2024-05-03T18:29:02.321 INFO:root:teuthology version: 0.0.1.dev271+g1ae7ad8 2024-05-03T18:29:02.321 DEBUG:teuthology.run:Teuthology command: teuthology --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/fsstress}} --verbose --name rishabh-2024-05-03_13:56:19-fs-wip-rishabh-testing-20240501.193033-testing-default-smithi --archive /home/teuthworker/archive/rishabh-2024-05-03_13:56:19-fs-wip-rishabh-testing-20240501.193033-testing-default-smithi/7687437 --owner scheduled_rishabh@teuthology -- /home/teuthworker/archive/rishabh-2024-05-03_13:56:19-fs-wip-rishabh-testing-20240501.193033-testing-default-smithi/7687437/orig.config.yaml 2024-05-03T18:29:02.434 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-03T18:29:02.547 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2024-05-03_13:56:19-fs-wip-rishabh-testing-20240501.193033-testing-default-smithi/7687437 branch: wip-rishabh-testing-20240501.193033 description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/fsstress}} email: null first_in_suite: false job_id: '7687437' kernel: client: sha1: distro flavor: default kdb: true sha1: 806ec36b21129f113ce356c007dd59694d340b37 ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: rishabh-2024-05-03_13:56:19-fs-wip-rishabh-testing-20240501.193033-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-rishabh-testing-20240501.193033 ceph: cephfs: max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 mds_export_ephemeral_random_max: 0.1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability sha1: 9a047fe7599a549042d06a3fb5b6813a203eabea subvols: create: 2 subvol_options: --namespace-isolated ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt install: ceph: flavor: default sha1: 9a047fe7599a549042d06a3fb5b6813a203eabea kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=crc - wsync syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-rishabh-testing-20240501.193033 sha1: 9a047fe7599a549042d06a3fb5b6813a203eabea owner: scheduled_rishabh@teuthology priority: 75 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 1698 sha1: 9a047fe7599a549042d06a3fb5b6813a203eabea sleep_before_teardown: 0 subset: 434/512 suite: fs suite_branch: wip-rishabh-testing-20240501.193033 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_9a047fe7599a549042d06a3fb5b6813a203eabea/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 9a047fe7599a549042d06a3fb5b6813a203eabea syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi032.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGxUx9zXAps/flzGKXz7J/qzDm7uw39KBO6vFgEzFxa043uqPPDmi+vp6QARnLnCM1JXCxrpJj+oM0z2v6/nYiw= smithi045.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCijT7gUF0xj4eI7WPYnoK1LtlFiIbP3NMtDWTAuZ70sh8iXe/89nzXF5017YKXNP037dwirklibLnlOnEPpI58= smithi113.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKsuRiYKdS/fvUtS0hCqaEOFNbV9STbiDwvuJcW4/SCPcs3UIfEMkqQy13DlSIxj+6j5J5sGEe2ASnnCZadPWGE= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - quiescer: null - workunit: clients: all: - suites/fsstress.sh timeout: 6h teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 1ae7ad82388e92a475afff437d49054826c019a1 timestamp: 2024-05-03_13:56:19 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-05-03T18:29:02.547 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_9a047fe7599a549042d06a3fb5b6813a203eabea/qa; will attempt to use it 2024-05-03T18:29:02.548 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_9a047fe7599a549042d06a3fb5b6813a203eabea/qa/tasks 2024-05-03T18:29:02.548 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-05-03T18:29:02.561 INFO:teuthology.task.internal:Checking packages... 2024-05-03T18:29:02.597 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '9a047fe7599a549042d06a3fb5b6813a203eabea' 2024-05-03T18:29:02.597 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-05-03T18:29:02.598 INFO:teuthology.packaging:ref: None 2024-05-03T18:29:02.598 INFO:teuthology.packaging:tag: None 2024-05-03T18:29:02.598 INFO:teuthology.packaging:branch: wip-rishabh-testing-20240501.193033 2024-05-03T18:29:02.598 INFO:teuthology.packaging:sha1: 9a047fe7599a549042d06a3fb5b6813a203eabea 2024-05-03T18:29:02.598 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-rishabh-testing-20240501.193033 2024-05-03T18:29:02.817 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3450.g9a047fe7 2024-05-03T18:29:02.818 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-05-03T18:29:02.848 INFO:teuthology.task.internal:no buildpackages task found 2024-05-03T18:29:02.849 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-05-03T18:29:02.912 INFO:teuthology.task.internal:Saving configuration 2024-05-03T18:29:02.934 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-05-03T18:29:03.159 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-03T18:29:03.185 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi032.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-05-03_13:56:19-fs-wip-rishabh-testing-20240501.193033-testing-default-smithi/7687437', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-03 18:23:22.550681', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGxUx9zXAps/flzGKXz7J/qzDm7uw39KBO6vFgEzFxa043uqPPDmi+vp6QARnLnCM1JXCxrpJj+oM0z2v6/nYiw='} 2024-05-03T18:29:03.207 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi045.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-05-03_13:56:19-fs-wip-rishabh-testing-20240501.193033-testing-default-smithi/7687437', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-03 18:23:22.551932', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCijT7gUF0xj4eI7WPYnoK1LtlFiIbP3NMtDWTAuZ70sh8iXe/89nzXF5017YKXNP037dwirklibLnlOnEPpI58='} 2024-05-03T18:29:03.228 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi113.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-05-03_13:56:19-fs-wip-rishabh-testing-20240501.193033-testing-default-smithi/7687437', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-03 18:23:22.549246', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKsuRiYKdS/fvUtS0hCqaEOFNbV9STbiDwvuJcW4/SCPcs3UIfEMkqQy13DlSIxj+6j5J5sGEe2ASnnCZadPWGE='} 2024-05-03T18:29:03.228 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-05-03T18:29:03.234 INFO:teuthology.task.internal:roles: ubuntu@smithi032.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-05-03T18:29:03.234 INFO:teuthology.task.internal:roles: ubuntu@smithi045.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-05-03T18:29:03.234 INFO:teuthology.task.internal:roles: ubuntu@smithi113.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-05-03T18:29:03.234 INFO:teuthology.run_tasks:Running task console_log... 2024-05-03T18:29:03.373 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fdb44735160>, signals=[15]) 2024-05-03T18:29:03.373 INFO:teuthology.run_tasks:Running task internal.connect... 2024-05-03T18:29:03.381 INFO:teuthology.task.internal:Opening connections... 2024-05-03T18:29:03.381 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi032.front.sepia.ceph.com 2024-05-03T18:29:03.383 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi032.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-03T18:29:03.464 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi045.front.sepia.ceph.com 2024-05-03T18:29:03.465 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi045.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-03T18:29:03.538 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi113.front.sepia.ceph.com 2024-05-03T18:29:03.539 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi113.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-03T18:29:03.616 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-05-03T18:29:03.622 DEBUG:teuthology.orchestra.run.smithi032:> uname -m 2024-05-03T18:29:03.641 INFO:teuthology.orchestra.run.smithi032.stdout:x86_64 2024-05-03T18:29:03.641 DEBUG:teuthology.orchestra.run.smithi032:> cat /etc/os-release 2024-05-03T18:29:03.699 INFO:teuthology.orchestra.run.smithi032.stdout:NAME="CentOS Stream" 2024-05-03T18:29:03.699 INFO:teuthology.orchestra.run.smithi032.stdout:VERSION="9" 2024-05-03T18:29:03.699 INFO:teuthology.orchestra.run.smithi032.stdout:ID="centos" 2024-05-03T18:29:03.699 INFO:teuthology.orchestra.run.smithi032.stdout:ID_LIKE="rhel fedora" 2024-05-03T18:29:03.699 INFO:teuthology.orchestra.run.smithi032.stdout:VERSION_ID="9" 2024-05-03T18:29:03.699 INFO:teuthology.orchestra.run.smithi032.stdout:PLATFORM_ID="platform:el9" 2024-05-03T18:29:03.699 INFO:teuthology.orchestra.run.smithi032.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-03T18:29:03.699 INFO:teuthology.orchestra.run.smithi032.stdout:ANSI_COLOR="0;31" 2024-05-03T18:29:03.699 INFO:teuthology.orchestra.run.smithi032.stdout:LOGO="fedora-logo-icon" 2024-05-03T18:29:03.700 INFO:teuthology.orchestra.run.smithi032.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-03T18:29:03.700 INFO:teuthology.orchestra.run.smithi032.stdout:HOME_URL="https://centos.org/" 2024-05-03T18:29:03.700 INFO:teuthology.orchestra.run.smithi032.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-03T18:29:03.700 INFO:teuthology.orchestra.run.smithi032.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-03T18:29:03.700 INFO:teuthology.orchestra.run.smithi032.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-03T18:29:03.700 INFO:teuthology.lock.ops:Updating smithi032.front.sepia.ceph.com on lock server 2024-05-03T18:29:03.725 DEBUG:teuthology.orchestra.run.smithi045:> uname -m 2024-05-03T18:29:03.743 INFO:teuthology.orchestra.run.smithi045.stdout:x86_64 2024-05-03T18:29:03.744 DEBUG:teuthology.orchestra.run.smithi045:> cat /etc/os-release 2024-05-03T18:29:03.802 INFO:teuthology.orchestra.run.smithi045.stdout:NAME="CentOS Stream" 2024-05-03T18:29:03.802 INFO:teuthology.orchestra.run.smithi045.stdout:VERSION="9" 2024-05-03T18:29:03.802 INFO:teuthology.orchestra.run.smithi045.stdout:ID="centos" 2024-05-03T18:29:03.802 INFO:teuthology.orchestra.run.smithi045.stdout:ID_LIKE="rhel fedora" 2024-05-03T18:29:03.802 INFO:teuthology.orchestra.run.smithi045.stdout:VERSION_ID="9" 2024-05-03T18:29:03.802 INFO:teuthology.orchestra.run.smithi045.stdout:PLATFORM_ID="platform:el9" 2024-05-03T18:29:03.802 INFO:teuthology.orchestra.run.smithi045.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-03T18:29:03.802 INFO:teuthology.orchestra.run.smithi045.stdout:ANSI_COLOR="0;31" 2024-05-03T18:29:03.802 INFO:teuthology.orchestra.run.smithi045.stdout:LOGO="fedora-logo-icon" 2024-05-03T18:29:03.803 INFO:teuthology.orchestra.run.smithi045.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-03T18:29:03.803 INFO:teuthology.orchestra.run.smithi045.stdout:HOME_URL="https://centos.org/" 2024-05-03T18:29:03.803 INFO:teuthology.orchestra.run.smithi045.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-03T18:29:03.803 INFO:teuthology.orchestra.run.smithi045.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-03T18:29:03.803 INFO:teuthology.orchestra.run.smithi045.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-03T18:29:03.803 INFO:teuthology.lock.ops:Updating smithi045.front.sepia.ceph.com on lock server 2024-05-03T18:29:03.825 DEBUG:teuthology.orchestra.run.smithi113:> uname -m 2024-05-03T18:29:03.843 INFO:teuthology.orchestra.run.smithi113.stdout:x86_64 2024-05-03T18:29:03.843 DEBUG:teuthology.orchestra.run.smithi113:> cat /etc/os-release 2024-05-03T18:29:03.899 INFO:teuthology.orchestra.run.smithi113.stdout:NAME="CentOS Stream" 2024-05-03T18:29:03.899 INFO:teuthology.orchestra.run.smithi113.stdout:VERSION="9" 2024-05-03T18:29:03.899 INFO:teuthology.orchestra.run.smithi113.stdout:ID="centos" 2024-05-03T18:29:03.899 INFO:teuthology.orchestra.run.smithi113.stdout:ID_LIKE="rhel fedora" 2024-05-03T18:29:03.900 INFO:teuthology.orchestra.run.smithi113.stdout:VERSION_ID="9" 2024-05-03T18:29:03.900 INFO:teuthology.orchestra.run.smithi113.stdout:PLATFORM_ID="platform:el9" 2024-05-03T18:29:03.900 INFO:teuthology.orchestra.run.smithi113.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-03T18:29:03.900 INFO:teuthology.orchestra.run.smithi113.stdout:ANSI_COLOR="0;31" 2024-05-03T18:29:03.900 INFO:teuthology.orchestra.run.smithi113.stdout:LOGO="fedora-logo-icon" 2024-05-03T18:29:03.900 INFO:teuthology.orchestra.run.smithi113.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-03T18:29:03.900 INFO:teuthology.orchestra.run.smithi113.stdout:HOME_URL="https://centos.org/" 2024-05-03T18:29:03.900 INFO:teuthology.orchestra.run.smithi113.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-03T18:29:03.900 INFO:teuthology.orchestra.run.smithi113.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-03T18:29:03.900 INFO:teuthology.orchestra.run.smithi113.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-03T18:29:03.900 INFO:teuthology.lock.ops:Updating smithi113.front.sepia.ceph.com on lock server 2024-05-03T18:29:03.925 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-05-03T18:29:03.936 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-05-03T18:29:04.018 INFO:teuthology.task.internal:Checking for old test directory... 2024-05-03T18:29:04.018 DEBUG:teuthology.orchestra.run.smithi032:> test '!' -e /home/ubuntu/cephtest 2024-05-03T18:29:04.021 DEBUG:teuthology.orchestra.run.smithi045:> test '!' -e /home/ubuntu/cephtest 2024-05-03T18:29:04.023 DEBUG:teuthology.orchestra.run.smithi113:> test '!' -e /home/ubuntu/cephtest 2024-05-03T18:29:04.039 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-05-03T18:29:04.046 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-05-03T18:29:04.046 DEBUG:teuthology.orchestra.run.smithi032:> test -z $(ls -A /var/lib/ceph) 2024-05-03T18:29:04.078 DEBUG:teuthology.orchestra.run.smithi045:> test -z $(ls -A /var/lib/ceph) 2024-05-03T18:29:04.080 DEBUG:teuthology.orchestra.run.smithi113:> test -z $(ls -A /var/lib/ceph) 2024-05-03T18:29:04.109 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-05-03T18:29:04.212 INFO:teuthology.run_tasks:Running task kernel... 2024-05-03T18:29:04.246 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'} 2024-05-03T18:29:04.246 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-05-03T18:29:04.246 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-05-03T18:29:04.247 DEBUG:teuthology.orchestra.run.smithi032:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-03T18:29:04.264 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-03T18:29:04.264 DEBUG:teuthology.orchestra.run.smithi032:> uname -r 2024-05-03T18:29:04.322 INFO:teuthology.orchestra.run.smithi032.stdout:5.14.0-437.el9.x86_64 2024-05-03T18:29:04.322 INFO:teuthology.task.kernel:Running kernel on smithi032: 5.14.0-437.el9.x86_64 2024-05-03T18:29:04.322 DEBUG:teuthology.orchestra.run.smithi032:> sudo yum install -y kernel 2024-05-03T18:29:07.547 INFO:teuthology.orchestra.run.smithi032.stdout:CentOS Stream 9 - BaseOS 4.9 MB/s | 8.0 MB 00:01 2024-05-03T18:29:09.348 INFO:teuthology.orchestra.run.smithi032.stdout:CentOS Stream 9 - AppStream 28 MB/s | 19 MB 00:00 2024-05-03T18:29:13.541 INFO:teuthology.orchestra.run.smithi032.stdout:CentOS Stream 9 - CRB 12 MB/s | 6.2 MB 00:00 2024-05-03T18:29:15.867 INFO:teuthology.orchestra.run.smithi032.stdout:CentOS Stream 9 - Extras packages 15 kB/s | 16 kB 00:01 2024-05-03T18:29:16.960 INFO:teuthology.orchestra.run.smithi032.stdout:Extra Packages for Enterprise Linux 22 MB/s | 21 MB 00:00 2024-05-03T18:29:22.227 INFO:teuthology.orchestra.run.smithi032.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-05-03T18:29:24.208 INFO:teuthology.orchestra.run.smithi032.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-05-03T18:29:24.208 INFO:teuthology.orchestra.run.smithi032.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-05-03T18:29:24.303 INFO:teuthology.orchestra.run.smithi032.stdout:Dependencies resolved. 2024-05-03T18:29:24.304 INFO:teuthology.orchestra.run.smithi032.stdout:Nothing to do. 2024-05-03T18:29:24.304 INFO:teuthology.orchestra.run.smithi032.stdout:Complete! 2024-05-03T18:29:24.404 DEBUG:teuthology.orchestra.run.smithi032:> echo no | sudo yum reinstall kernel || true 2024-05-03T18:29:24.925 INFO:teuthology.orchestra.run.smithi032.stdout:Last metadata expiration check: 0:00:02 ago on Fri 03 May 2024 06:29:22 PM UTC. 2024-05-03T18:29:25.049 INFO:teuthology.orchestra.run.smithi032.stdout:Dependencies resolved. 2024-05-03T18:29:25.063 INFO:teuthology.orchestra.run.smithi032.stdout:================================================================================ 2024-05-03T18:29:25.063 INFO:teuthology.orchestra.run.smithi032.stdout: Package Architecture Version Repository Size 2024-05-03T18:29:25.063 INFO:teuthology.orchestra.run.smithi032.stdout:================================================================================ 2024-05-03T18:29:25.063 INFO:teuthology.orchestra.run.smithi032.stdout:Reinstalling: 2024-05-03T18:29:25.063 INFO:teuthology.orchestra.run.smithi032.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-05-03T18:29:25.063 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-03T18:29:25.063 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction Summary 2024-05-03T18:29:25.063 INFO:teuthology.orchestra.run.smithi032.stdout:================================================================================ 2024-05-03T18:29:25.063 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-03T18:29:25.064 INFO:teuthology.orchestra.run.smithi032.stdout:Total download size: 255 k 2024-05-03T18:29:25.064 INFO:teuthology.orchestra.run.smithi032.stdout:Installed size: 0 2024-05-03T18:29:25.064 INFO:teuthology.orchestra.run.smithi032.stderr:Operation aborted. 2024-05-03T18:29:25.124 INFO:teuthology.orchestra.run.smithi032.stdout:Is this ok [y/N]: 2024-05-03T18:29:25.124 DEBUG:teuthology.orchestra.run.smithi032:> sudo yum reinstall -y kernel || true 2024-05-03T18:29:25.640 INFO:teuthology.orchestra.run.smithi032.stdout:Last metadata expiration check: 0:00:03 ago on Fri 03 May 2024 06:29:22 PM UTC. 2024-05-03T18:29:25.760 INFO:teuthology.orchestra.run.smithi032.stdout:Dependencies resolved. 2024-05-03T18:29:25.761 INFO:teuthology.orchestra.run.smithi032.stdout:================================================================================ 2024-05-03T18:29:25.761 INFO:teuthology.orchestra.run.smithi032.stdout: Package Architecture Version Repository Size 2024-05-03T18:29:25.761 INFO:teuthology.orchestra.run.smithi032.stdout:================================================================================ 2024-05-03T18:29:25.761 INFO:teuthology.orchestra.run.smithi032.stdout:Reinstalling: 2024-05-03T18:29:25.761 INFO:teuthology.orchestra.run.smithi032.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-05-03T18:29:25.761 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-03T18:29:25.761 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction Summary 2024-05-03T18:29:25.762 INFO:teuthology.orchestra.run.smithi032.stdout:================================================================================ 2024-05-03T18:29:25.762 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-03T18:29:25.762 INFO:teuthology.orchestra.run.smithi032.stdout:Total download size: 255 k 2024-05-03T18:29:25.762 INFO:teuthology.orchestra.run.smithi032.stdout:Installed size: 0 2024-05-03T18:29:25.762 INFO:teuthology.orchestra.run.smithi032.stdout:Downloading Packages: 2024-05-03T18:29:26.479 INFO:teuthology.orchestra.run.smithi032.stdout:kernel-5.14.0-437.el9.x86_64.rpm 428 kB/s | 255 kB 00:00 2024-05-03T18:29:26.479 INFO:teuthology.orchestra.run.smithi032.stdout:-------------------------------------------------------------------------------- 2024-05-03T18:29:26.479 INFO:teuthology.orchestra.run.smithi032.stdout:Total 355 kB/s | 255 kB 00:00 2024-05-03T18:29:26.507 INFO:teuthology.orchestra.run.smithi032.stdout:Running transaction check 2024-05-03T18:29:26.704 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction check succeeded. 2024-05-03T18:29:26.704 INFO:teuthology.orchestra.run.smithi032.stdout:Running transaction test 2024-05-03T18:29:26.709 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction test succeeded. 2024-05-03T18:29:26.710 INFO:teuthology.orchestra.run.smithi032.stdout:Running transaction 2024-05-03T18:29:26.865 INFO:teuthology.orchestra.run.smithi032.stdout: Preparing : 1/1 2024-05-03T18:29:26.938 INFO:teuthology.orchestra.run.smithi032.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-05-03T18:29:26.938 INFO:teuthology.orchestra.run.smithi032.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-05-03T18:29:27.639 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-05-03T18:29:27.639 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-05-03T18:29:28.344 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-05-03T18:29:28.344 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-03T18:29:28.344 INFO:teuthology.orchestra.run.smithi032.stdout:Reinstalled: 2024-05-03T18:29:28.344 INFO:teuthology.orchestra.run.smithi032.stdout: kernel-5.14.0-437.el9.x86_64 2024-05-03T18:29:28.344 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-03T18:29:28.344 INFO:teuthology.orchestra.run.smithi032.stdout:Complete! 2024-05-03T18:29:28.525 DEBUG:teuthology.orchestra.run.smithi032:> rpm -q kernel | sort -rV | head -n 1 2024-05-03T18:29:28.574 INFO:teuthology.orchestra.run.smithi032.stdout:kernel-5.14.0-437.el9.x86_64 2024-05-03T18:29:28.574 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-05-03T18:29:28.575 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-05-03T18:29:28.575 DEBUG:teuthology.parallel:result is None 2024-05-03T18:29:28.575 INFO:teuthology.run_tasks:Running task internal.base... 2024-05-03T18:29:28.583 INFO:teuthology.task.internal:Creating test directory... 2024-05-03T18:29:28.583 DEBUG:teuthology.orchestra.run.smithi032:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-03T18:29:28.586 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-03T18:29:28.588 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-03T18:29:28.605 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-05-03T18:29:28.640 INFO:teuthology.run_tasks:Running task internal.archive... 2024-05-03T18:29:28.703 INFO:teuthology.task.internal:Creating archive directory... 2024-05-03T18:29:28.703 DEBUG:teuthology.orchestra.run.smithi032:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-03T18:29:28.705 DEBUG:teuthology.orchestra.run.smithi045:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-03T18:29:28.708 DEBUG:teuthology.orchestra.run.smithi113:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-03T18:29:28.747 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-05-03T18:29:28.774 INFO:teuthology.task.internal:Enabling coredump saving... 2024-05-03T18:29:28.775 DEBUG:teuthology.orchestra.run.smithi045:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-03T18:29:28.791 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-03T18:29:28.792 DEBUG:teuthology.orchestra.run.smithi113:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-03T18:29:28.808 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-03T18:29:28.809 DEBUG:teuthology.orchestra.run.smithi032:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-03T18:29:28.811 DEBUG:teuthology.orchestra.run.smithi045:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-03T18:29:28.834 DEBUG:teuthology.orchestra.run.smithi113:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-03T18:29:28.858 INFO:teuthology.orchestra.run.smithi032.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-03T18:29:28.868 INFO:teuthology.orchestra.run.smithi045.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-03T18:29:28.870 INFO:teuthology.orchestra.run.smithi032.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-03T18:29:28.882 INFO:teuthology.orchestra.run.smithi045.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-03T18:29:28.884 INFO:teuthology.orchestra.run.smithi113.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-03T18:29:28.914 INFO:teuthology.orchestra.run.smithi113.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-03T18:29:28.916 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-05-03T18:29:28.923 INFO:teuthology.task.internal:Configuring sudo... 2024-05-03T18:29:28.923 DEBUG:teuthology.orchestra.run.smithi032:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-03T18:29:28.926 DEBUG:teuthology.orchestra.run.smithi045:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-03T18:29:28.928 DEBUG:teuthology.orchestra.run.smithi113:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-03T18:29:28.982 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-05-03T18:29:28.991 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-05-03T18:29:28.992 DEBUG:teuthology.orchestra.run.smithi032:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-03T18:29:28.994 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-03T18:29:28.997 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-03T18:29:29.039 DEBUG:teuthology.orchestra.run.smithi032:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-03T18:29:29.095 DEBUG:teuthology.orchestra.run.smithi032:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-03T18:29:29.139 DEBUG:teuthology.orchestra.run.smithi032:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-03T18:29:29.222 DEBUG:teuthology.orchestra.run.smithi032:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-03T18:29:29.261 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-05-03T18:29:29.262 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-03T18:29:29.326 DEBUG:teuthology.orchestra.run.smithi045:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-03T18:29:29.370 DEBUG:teuthology.orchestra.run.smithi045:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-03T18:29:29.427 DEBUG:teuthology.orchestra.run.smithi045:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-03T18:29:29.511 DEBUG:teuthology.orchestra.run.smithi045:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-03T18:29:29.554 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-05-03T18:29:29.555 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-03T18:29:29.621 DEBUG:teuthology.orchestra.run.smithi113:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-03T18:29:29.669 DEBUG:teuthology.orchestra.run.smithi113:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-03T18:29:29.715 DEBUG:teuthology.orchestra.run.smithi113:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-03T18:29:29.799 DEBUG:teuthology.orchestra.run.smithi113:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-03T18:29:29.842 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-05-03T18:29:29.842 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-03T18:29:29.908 DEBUG:teuthology.orchestra.run.smithi032:> sudo service rsyslog restart 2024-05-03T18:29:29.910 DEBUG:teuthology.orchestra.run.smithi045:> sudo service rsyslog restart 2024-05-03T18:29:29.913 DEBUG:teuthology.orchestra.run.smithi113:> sudo service rsyslog restart 2024-05-03T18:29:29.951 INFO:teuthology.orchestra.run.smithi032.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-03T18:29:29.954 INFO:teuthology.orchestra.run.smithi045.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-03T18:29:29.988 INFO:teuthology.orchestra.run.smithi113.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-03T18:29:30.355 INFO:teuthology.run_tasks:Running task internal.timer... 2024-05-03T18:29:30.362 INFO:teuthology.task.internal:Starting timer... 2024-05-03T18:29:30.362 INFO:teuthology.run_tasks:Running task pcp... 2024-05-03T18:29:30.379 INFO:teuthology.run_tasks:Running task selinux... 2024-05-03T18:29:30.436 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-05-03T18:29:30.436 DEBUG:teuthology.orchestra.run.smithi032:> sudo service auditd rotate 2024-05-03T18:29:30.500 INFO:teuthology.orchestra.run.smithi032.stdout:Rotating logs: 2024-05-03T18:29:30.502 DEBUG:teuthology.orchestra.run.smithi045:> sudo service auditd rotate 2024-05-03T18:29:30.568 INFO:teuthology.orchestra.run.smithi045.stdout:Rotating logs: 2024-05-03T18:29:30.570 DEBUG:teuthology.orchestra.run.smithi113:> sudo service auditd rotate 2024-05-03T18:29:30.625 INFO:teuthology.orchestra.run.smithi113.stdout:Rotating logs: 2024-05-03T18:29:30.627 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-05-03T18:29:30.627 DEBUG:teuthology.orchestra.run.smithi032:> /usr/sbin/getenforce 2024-05-03T18:29:30.649 INFO:teuthology.orchestra.run.smithi032.stdout:Permissive 2024-05-03T18:29:30.649 DEBUG:teuthology.orchestra.run.smithi045:> /usr/sbin/getenforce 2024-05-03T18:29:30.675 INFO:teuthology.orchestra.run.smithi045.stdout:Permissive 2024-05-03T18:29:30.675 DEBUG:teuthology.orchestra.run.smithi113:> /usr/sbin/getenforce 2024-05-03T18:29:30.699 INFO:teuthology.orchestra.run.smithi113.stdout:Permissive 2024-05-03T18:29:30.699 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi032.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi045.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi113.front.sepia.ceph.com': 'permissive'} 2024-05-03T18:29:30.699 DEBUG:teuthology.orchestra.run.smithi032:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-03T18:29:30.725 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-03T18:29:30.726 DEBUG:teuthology.orchestra.run.smithi045:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-03T18:29:30.754 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-03T18:29:30.754 DEBUG:teuthology.orchestra.run.smithi113:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-03T18:29:30.780 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-03T18:29:30.781 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-05-03T18:29:30.781 DEBUG:teuthology.orchestra.run.smithi032:> sudo /usr/sbin/setenforce permissive 2024-05-03T18:29:30.806 DEBUG:teuthology.orchestra.run.smithi045:> sudo /usr/sbin/setenforce permissive 2024-05-03T18:29:30.835 DEBUG:teuthology.orchestra.run.smithi113:> sudo /usr/sbin/setenforce permissive 2024-05-03T18:29:30.860 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-05-03T18:29:30.870 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-05-03T18:29:30.887 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-05-03T18:29:30.888 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi032.front.sepia.ceph.com,smithi045.front.sepia.ceph.com,smithi113.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-05-03T18:34:00.172 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi032.front.sepia.ceph.com'), Remote(name='ubuntu@smithi045.front.sepia.ceph.com'), Remote(name='ubuntu@smithi113.front.sepia.ceph.com')] 2024-05-03T18:34:00.173 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi032.front.sepia.ceph.com' 2024-05-03T18:34:00.175 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi032.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-03T18:34:00.254 DEBUG:teuthology.orchestra.run.smithi032:> true 2024-05-03T18:34:00.335 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi032.front.sepia.ceph.com' 2024-05-03T18:34:00.335 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi045.front.sepia.ceph.com' 2024-05-03T18:34:00.336 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi045.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-03T18:34:00.410 DEBUG:teuthology.orchestra.run.smithi045:> true 2024-05-03T18:34:00.491 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi045.front.sepia.ceph.com' 2024-05-03T18:34:00.491 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi113.front.sepia.ceph.com' 2024-05-03T18:34:00.492 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi113.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-03T18:34:00.568 DEBUG:teuthology.orchestra.run.smithi113:> true 2024-05-03T18:34:00.646 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi113.front.sepia.ceph.com' 2024-05-03T18:34:00.647 INFO:teuthology.run_tasks:Running task clock... 2024-05-03T18:34:00.660 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-05-03T18:34:00.660 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-03T18:34:00.661 DEBUG:teuthology.orchestra.run.smithi032:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-03T18:34:00.663 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-03T18:34:00.663 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-03T18:34:00.666 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-03T18:34:00.666 DEBUG:teuthology.orchestra.run.smithi113:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-03T18:34:00.694 INFO:teuthology.orchestra.run.smithi032.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-03T18:34:00.697 INFO:teuthology.orchestra.run.smithi045.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-03T18:34:00.711 INFO:teuthology.orchestra.run.smithi032.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-03T18:34:00.712 INFO:teuthology.orchestra.run.smithi045.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-03T18:34:00.718 INFO:teuthology.orchestra.run.smithi113.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-03T18:34:00.732 INFO:teuthology.orchestra.run.smithi113.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-03T18:34:00.737 INFO:teuthology.orchestra.run.smithi032.stderr:sudo: ntpd: command not found 2024-05-03T18:34:00.738 INFO:teuthology.orchestra.run.smithi045.stderr:sudo: ntpd: command not found 2024-05-03T18:34:00.748 INFO:teuthology.orchestra.run.smithi032.stdout:506 Cannot talk to daemon 2024-05-03T18:34:00.749 INFO:teuthology.orchestra.run.smithi045.stdout:506 Cannot talk to daemon 2024-05-03T18:34:00.756 INFO:teuthology.orchestra.run.smithi113.stderr:sudo: ntpd: command not found 2024-05-03T18:34:00.763 INFO:teuthology.orchestra.run.smithi032.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-03T18:34:00.764 INFO:teuthology.orchestra.run.smithi045.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-03T18:34:00.767 INFO:teuthology.orchestra.run.smithi113.stdout:506 Cannot talk to daemon 2024-05-03T18:34:00.777 INFO:teuthology.orchestra.run.smithi032.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-03T18:34:00.778 INFO:teuthology.orchestra.run.smithi045.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-03T18:34:00.780 INFO:teuthology.orchestra.run.smithi113.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-03T18:34:00.793 INFO:teuthology.orchestra.run.smithi113.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-03T18:34:00.835 INFO:teuthology.orchestra.run.smithi045.stderr:bash: line 1: ntpq: command not found 2024-05-03T18:34:00.838 INFO:teuthology.orchestra.run.smithi045.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-03T18:34:00.838 INFO:teuthology.orchestra.run.smithi045.stdout:=============================================================================== 2024-05-03T18:34:00.838 INFO:teuthology.orchestra.run.smithi045.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-03T18:34:00.839 INFO:teuthology.orchestra.run.smithi045.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-03T18:34:00.839 INFO:teuthology.orchestra.run.smithi045.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-03T18:34:00.839 INFO:teuthology.orchestra.run.smithi045.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-03T18:34:00.840 INFO:teuthology.orchestra.run.smithi032.stderr:bash: line 1: ntpq: command not found 2024-05-03T18:34:00.842 INFO:teuthology.orchestra.run.smithi032.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-03T18:34:00.842 INFO:teuthology.orchestra.run.smithi032.stdout:=============================================================================== 2024-05-03T18:34:00.842 INFO:teuthology.orchestra.run.smithi032.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-03T18:34:00.843 INFO:teuthology.orchestra.run.smithi032.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-03T18:34:00.843 INFO:teuthology.orchestra.run.smithi032.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-03T18:34:00.843 INFO:teuthology.orchestra.run.smithi032.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-03T18:34:00.851 INFO:teuthology.orchestra.run.smithi113.stderr:bash: line 1: ntpq: command not found 2024-05-03T18:34:00.854 INFO:teuthology.orchestra.run.smithi113.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-03T18:34:00.854 INFO:teuthology.orchestra.run.smithi113.stdout:=============================================================================== 2024-05-03T18:34:00.855 INFO:teuthology.orchestra.run.smithi113.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-03T18:34:00.855 INFO:teuthology.orchestra.run.smithi113.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-03T18:34:00.855 INFO:teuthology.orchestra.run.smithi113.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-03T18:34:00.855 INFO:teuthology.orchestra.run.smithi113.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-03T18:34:00.856 INFO:teuthology.run_tasks:Running task install... 2024-05-03T18:34:00.865 DEBUG:teuthology.task.install:project ceph 2024-05-03T18:34:00.865 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '9a047fe7599a549042d06a3fb5b6813a203eabea'}} 2024-05-03T18:34:00.865 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '9a047fe7599a549042d06a3fb5b6813a203eabea'} 2024-05-03T18:34:00.865 INFO:teuthology.task.install:Using flavor: default 2024-05-03T18:34:00.871 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-05-03T18:34:00.871 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-05-03T18:34:00.871 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '9a047fe7599a549042d06a3fb5b6813a203eabea', 'tag': None, 'wait_for_package': False} 2024-05-03T18:34:00.872 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=9a047fe7599a549042d06a3fb5b6813a203eabea 2024-05-03T18:34:00.874 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '9a047fe7599a549042d06a3fb5b6813a203eabea', 'tag': None, 'wait_for_package': False} 2024-05-03T18:34:00.874 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=9a047fe7599a549042d06a3fb5b6813a203eabea 2024-05-03T18:34:00.875 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '9a047fe7599a549042d06a3fb5b6813a203eabea', 'tag': None, 'wait_for_package': False} 2024-05-03T18:34:00.875 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=9a047fe7599a549042d06a3fb5b6813a203eabea 2024-05-03T18:34:01.035 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240501.193033/9a047fe7599a549042d06a3fb5b6813a203eabea/centos/9/flavors/default/ 2024-05-03T18:34:01.035 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3450.g9a047fe7 2024-05-03T18:34:01.103 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240501.193033/9a047fe7599a549042d06a3fb5b6813a203eabea/centos/9/flavors/default/ 2024-05-03T18:34:01.103 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3450.g9a047fe7 2024-05-03T18:34:01.108 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240501.193033/9a047fe7599a549042d06a3fb5b6813a203eabea/centos/9/flavors/default/ 2024-05-03T18:34:01.108 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3450.g9a047fe7 2024-05-03T18:34:01.152 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240501.193033/9a047fe7599a549042d06a3fb5b6813a203eabea/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240501.193033/9a047fe7599a549042d06a3fb5b6813a203eabea/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240501.193033/9a047fe7599a549042d06a3fb5b6813a203eabea/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-03T18:34:01.153 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-05-03T18:34:01.153 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-03T18:34:01.180 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-03T18:34:01.181 DEBUG:teuthology.orchestra.run.smithi032:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/9a047fe7599a549042d06a3fb5b6813a203eabea/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-03T18:34:01.219 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240501.193033/9a047fe7599a549042d06a3fb5b6813a203eabea/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240501.193033/9a047fe7599a549042d06a3fb5b6813a203eabea/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240501.193033/9a047fe7599a549042d06a3fb5b6813a203eabea/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-03T18:34:01.219 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-05-03T18:34:01.219 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-03T18:34:01.244 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-03T18:34:01.245 DEBUG:teuthology.orchestra.run.smithi113:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/9a047fe7599a549042d06a3fb5b6813a203eabea/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-03T18:34:01.248 DEBUG:teuthology.orchestra.run.smithi032:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-03T18:34:01.273 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240501.193033/9a047fe7599a549042d06a3fb5b6813a203eabea/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240501.193033/9a047fe7599a549042d06a3fb5b6813a203eabea/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240501.193033/9a047fe7599a549042d06a3fb5b6813a203eabea/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-03T18:34:01.273 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-05-03T18:34:01.273 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-03T18:34:01.302 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-03T18:34:01.302 DEBUG:teuthology.orchestra.run.smithi045:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/9a047fe7599a549042d06a3fb5b6813a203eabea/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-03T18:34:01.309 DEBUG:teuthology.orchestra.run.smithi113:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-03T18:34:01.323 DEBUG:teuthology.orchestra.run.smithi032:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-03T18:34:01.369 DEBUG:teuthology.orchestra.run.smithi045:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-03T18:34:01.381 DEBUG:teuthology.orchestra.run.smithi113:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-03T18:34:01.387 INFO:teuthology.orchestra.run.smithi032.stdout:check_obsoletes = 1 2024-05-03T18:34:01.389 DEBUG:teuthology.orchestra.run.smithi032:> sudo yum clean all 2024-05-03T18:34:01.444 INFO:teuthology.orchestra.run.smithi113.stdout:check_obsoletes = 1 2024-05-03T18:34:01.446 DEBUG:teuthology.orchestra.run.smithi113:> sudo yum clean all 2024-05-03T18:34:01.448 DEBUG:teuthology.orchestra.run.smithi045:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-03T18:34:01.514 INFO:teuthology.orchestra.run.smithi045.stdout:check_obsoletes = 1 2024-05-03T18:34:01.516 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum clean all 2024-05-03T18:34:01.623 INFO:teuthology.orchestra.run.smithi032.stdout:50 files removed 2024-05-03T18:34:01.651 DEBUG:teuthology.orchestra.run.smithi032:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-03T18:34:01.671 INFO:teuthology.orchestra.run.smithi113.stdout:50 files removed 2024-05-03T18:34:01.699 DEBUG:teuthology.orchestra.run.smithi113:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-03T18:34:01.756 INFO:teuthology.orchestra.run.smithi045.stdout:50 files removed 2024-05-03T18:34:01.784 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-03T18:34:02.493 INFO:teuthology.orchestra.run.smithi045.stdout:ceph packages for x86_64 194 kB/s | 84 kB 00:00 2024-05-03T18:34:02.634 INFO:teuthology.orchestra.run.smithi113.stdout:ceph packages for x86_64 124 kB/s | 84 kB 00:00 2024-05-03T18:34:02.769 INFO:teuthology.orchestra.run.smithi032.stdout:ceph packages for x86_64 99 kB/s | 84 kB 00:00 2024-05-03T18:34:02.861 INFO:teuthology.orchestra.run.smithi045.stdout:ceph noarch packages 37 kB/s | 12 kB 00:00 2024-05-03T18:34:03.077 INFO:teuthology.orchestra.run.smithi113.stdout:ceph noarch packages 29 kB/s | 12 kB 00:00 2024-05-03T18:34:03.153 INFO:teuthology.orchestra.run.smithi032.stdout:ceph noarch packages 35 kB/s | 12 kB 00:00 2024-05-03T18:34:03.294 INFO:teuthology.orchestra.run.smithi045.stdout:ceph source packages 4.8 kB/s | 1.9 kB 00:00 2024-05-03T18:34:03.579 INFO:teuthology.orchestra.run.smithi032.stdout:ceph source packages 4.9 kB/s | 1.9 kB 00:00 2024-05-03T18:34:03.686 INFO:teuthology.orchestra.run.smithi113.stdout:ceph source packages 3.3 kB/s | 1.9 kB 00:00 2024-05-03T18:34:04.678 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS Stream 9 - BaseOS 6.0 MB/s | 8.0 MB 00:01 2024-05-03T18:34:04.885 INFO:teuthology.orchestra.run.smithi113.stdout:CentOS Stream 9 - BaseOS 6.9 MB/s | 8.0 MB 00:01 2024-05-03T18:34:05.504 INFO:teuthology.orchestra.run.smithi032.stdout:CentOS Stream 9 - BaseOS 4.3 MB/s | 8.0 MB 00:01 2024-05-03T18:34:06.520 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS Stream 9 - AppStream 27 MB/s | 19 MB 00:00 2024-05-03T18:34:07.313 INFO:teuthology.orchestra.run.smithi032.stdout:CentOS Stream 9 - AppStream 27 MB/s | 19 MB 00:00 2024-05-03T18:34:08.978 INFO:teuthology.orchestra.run.smithi113.stdout:CentOS Stream 9 - AppStream 6.3 MB/s | 19 MB 00:03 2024-05-03T18:34:10.679 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS Stream 9 - CRB 14 MB/s | 6.2 MB 00:00 2024-05-03T18:34:11.639 INFO:teuthology.orchestra.run.smithi032.stdout:CentOS Stream 9 - CRB 10 MB/s | 6.2 MB 00:00 2024-05-03T18:34:12.197 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS Stream 9 - Extras packages 54 kB/s | 16 kB 00:00 2024-05-03T18:34:13.248 INFO:teuthology.orchestra.run.smithi032.stdout:CentOS Stream 9 - Extras packages 41 kB/s | 16 kB 00:00 2024-05-03T18:34:14.103 INFO:teuthology.orchestra.run.smithi113.stdout:CentOS Stream 9 - CRB 3.9 MB/s | 6.2 MB 00:01 2024-05-03T18:34:14.367 INFO:teuthology.orchestra.run.smithi032.stdout:Extra Packages for Enterprise Linux 21 MB/s | 21 MB 00:00 2024-05-03T18:34:15.688 INFO:teuthology.orchestra.run.smithi113.stdout:CentOS Stream 9 - Extras packages 36 kB/s | 16 kB 00:00 2024-05-03T18:34:16.548 INFO:teuthology.orchestra.run.smithi045.stdout:Extra Packages for Enterprise Linux 5.0 MB/s | 21 MB 00:04 2024-05-03T18:34:17.122 INFO:teuthology.orchestra.run.smithi113.stdout:Extra Packages for Enterprise Linux 16 MB/s | 21 MB 00:01 2024-05-03T18:34:19.675 INFO:teuthology.orchestra.run.smithi032.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-05-03T18:34:21.878 INFO:teuthology.orchestra.run.smithi032.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-03T18:34:21.878 INFO:teuthology.orchestra.run.smithi032.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-03T18:34:21.889 INFO:teuthology.orchestra.run.smithi032.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-03T18:34:21.890 INFO:teuthology.orchestra.run.smithi045.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-05-03T18:34:21.890 INFO:teuthology.orchestra.run.smithi032.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-03T18:34:21.891 INFO:teuthology.orchestra.run.smithi032.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-03T18:34:21.892 INFO:teuthology.orchestra.run.smithi032.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-03T18:34:21.893 INFO:teuthology.orchestra.run.smithi032.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-03T18:34:21.896 INFO:teuthology.orchestra.run.smithi032.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-03T18:34:21.897 INFO:teuthology.orchestra.run.smithi032.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-03T18:34:21.961 INFO:teuthology.orchestra.run.smithi032.stdout:Dependencies resolved. 2024-05-03T18:34:21.970 INFO:teuthology.orchestra.run.smithi032.stdout:======================================================================================= 2024-05-03T18:34:21.970 INFO:teuthology.orchestra.run.smithi032.stdout: Package Arch Version Repository Size 2024-05-03T18:34:21.971 INFO:teuthology.orchestra.run.smithi032.stdout:======================================================================================= 2024-05-03T18:34:21.971 INFO:teuthology.orchestra.run.smithi032.stdout:Installing: 2024-05-03T18:34:21.971 INFO:teuthology.orchestra.run.smithi032.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-03T18:34:21.971 INFO:teuthology.orchestra.run.smithi032.stdout: ceph x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 6.6 k 2024-05-03T18:34:21.971 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-base x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 5.4 M 2024-05-03T18:34:21.971 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-fuse x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 891 k 2024-05-03T18:34:21.971 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 145 k 2024-05-03T18:34:21.971 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 1.5 M 2024-05-03T18:34:21.971 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3450.g9a047fe7.el9 ceph-noarch 142 k 2024-05-03T18:34:21.971 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3450.g9a047fe7.el9 ceph-noarch 3.5 M 2024-05-03T18:34:21.971 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3450.g9a047fe7.el9 ceph-noarch 7.4 M 2024-05-03T18:34:21.971 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-rook noarch 2:19.0.0-3450.g9a047fe7.el9 ceph-noarch 49 k 2024-05-03T18:34:21.972 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-radosgw x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 11 M 2024-05-03T18:34:21.972 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-test x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 49 M 2024-05-03T18:34:21.972 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-volume noarch 2:19.0.0-3450.g9a047fe7.el9 ceph-noarch 273 k 2024-05-03T18:34:21.972 INFO:teuthology.orchestra.run.smithi032.stdout: cephadm noarch 2:19.0.0-3450.g9a047fe7.el9 ceph-noarch 763 k 2024-05-03T18:34:21.972 INFO:teuthology.orchestra.run.smithi032.stdout: cephfs-mirror x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 225 k 2024-05-03T18:34:21.972 INFO:teuthology.orchestra.run.smithi032.stdout: cephfs-shell x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 27 k 2024-05-03T18:34:21.972 INFO:teuthology.orchestra.run.smithi032.stdout: cephfs-top noarch 2:19.0.0-3450.g9a047fe7.el9 ceph-noarch 18 k 2024-05-03T18:34:21.972 INFO:teuthology.orchestra.run.smithi032.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-05-03T18:34:21.972 INFO:teuthology.orchestra.run.smithi032.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-03T18:34:21.972 INFO:teuthology.orchestra.run.smithi032.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-03T18:34:21.972 INFO:teuthology.orchestra.run.smithi032.stdout: libcephfs-devel x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 32 k 2024-05-03T18:34:21.972 INFO:teuthology.orchestra.run.smithi032.stdout: libcephfs2 x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 747 k 2024-05-03T18:34:21.972 INFO:teuthology.orchestra.run.smithi032.stdout: librados-devel x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 127 k 2024-05-03T18:34:21.973 INFO:teuthology.orchestra.run.smithi032.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-03T18:34:21.973 INFO:teuthology.orchestra.run.smithi032.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-03T18:34:21.973 INFO:teuthology.orchestra.run.smithi032.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-03T18:34:21.973 INFO:teuthology.orchestra.run.smithi032.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-03T18:34:21.973 INFO:teuthology.orchestra.run.smithi032.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-03T18:34:21.973 INFO:teuthology.orchestra.run.smithi032.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-03T18:34:21.973 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cephfs x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 161 k 2024-05-03T18:34:21.973 INFO:teuthology.orchestra.run.smithi032.stdout: python3-rados x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 320 k 2024-05-03T18:34:21.973 INFO:teuthology.orchestra.run.smithi032.stdout: python3-rbd x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 298 k 2024-05-03T18:34:21.973 INFO:teuthology.orchestra.run.smithi032.stdout: python3-rgw x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 100 k 2024-05-03T18:34:21.973 INFO:teuthology.orchestra.run.smithi032.stdout: rbd-fuse x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 85 k 2024-05-03T18:34:21.973 INFO:teuthology.orchestra.run.smithi032.stdout: rbd-mirror x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 3.1 M 2024-05-03T18:34:21.973 INFO:teuthology.orchestra.run.smithi032.stdout: rbd-nbd x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 170 k 2024-05-03T18:34:21.974 INFO:teuthology.orchestra.run.smithi032.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-03T18:34:21.974 INFO:teuthology.orchestra.run.smithi032.stdout:Upgrading: 2024-05-03T18:34:21.974 INFO:teuthology.orchestra.run.smithi032.stdout: librados2 x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 3.4 M 2024-05-03T18:34:21.974 INFO:teuthology.orchestra.run.smithi032.stdout: librbd1 x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 3.1 M 2024-05-03T18:34:21.974 INFO:teuthology.orchestra.run.smithi032.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-03T18:34:21.974 INFO:teuthology.orchestra.run.smithi032.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-03T18:34:21.974 INFO:teuthology.orchestra.run.smithi032.stdout:Installing dependencies: 2024-05-03T18:34:21.974 INFO:teuthology.orchestra.run.smithi032.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-03T18:34:21.974 INFO:teuthology.orchestra.run.smithi032.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-03T18:34:21.974 INFO:teuthology.orchestra.run.smithi032.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-03T18:34:21.974 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-common x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 21 M 2024-05-03T18:34:21.974 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3450.g9a047fe7.el9 ceph-noarch 34 k 2024-05-03T18:34:21.974 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mds x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 2.4 M 2024-05-03T18:34:21.974 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3450.g9a047fe7.el9 ceph-noarch 274 k 2024-05-03T18:34:21.975 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mon x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 4.7 M 2024-05-03T18:34:21.975 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-osd x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 17 M 2024-05-03T18:34:21.975 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3450.g9a047fe7.el9 ceph-noarch 16 k 2024-05-03T18:34:21.975 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-selinux x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 25 k 2024-05-03T18:34:21.975 INFO:teuthology.orchestra.run.smithi032.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-03T18:34:21.975 INFO:teuthology.orchestra.run.smithi032.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-03T18:34:21.975 INFO:teuthology.orchestra.run.smithi032.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-03T18:34:21.975 INFO:teuthology.orchestra.run.smithi032.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-03T18:34:21.975 INFO:teuthology.orchestra.run.smithi032.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-03T18:34:21.975 INFO:teuthology.orchestra.run.smithi032.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-03T18:34:21.975 INFO:teuthology.orchestra.run.smithi032.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-03T18:34:21.975 INFO:teuthology.orchestra.run.smithi032.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-03T18:34:21.975 INFO:teuthology.orchestra.run.smithi032.stdout: libcephsqlite x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 163 k 2024-05-03T18:34:21.975 INFO:teuthology.orchestra.run.smithi032.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-03T18:34:21.976 INFO:teuthology.orchestra.run.smithi032.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-03T18:34:21.976 INFO:teuthology.orchestra.run.smithi032.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-03T18:34:21.976 INFO:teuthology.orchestra.run.smithi032.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-03T18:34:21.976 INFO:teuthology.orchestra.run.smithi032.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-03T18:34:21.976 INFO:teuthology.orchestra.run.smithi032.stdout: libradosstriper1 x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 500 k 2024-05-03T18:34:21.976 INFO:teuthology.orchestra.run.smithi032.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-03T18:34:21.976 INFO:teuthology.orchestra.run.smithi032.stdout: librgw2 x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 5.4 M 2024-05-03T18:34:21.976 INFO:teuthology.orchestra.run.smithi032.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-03T18:34:21.976 INFO:teuthology.orchestra.run.smithi032.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-03T18:34:21.976 INFO:teuthology.orchestra.run.smithi032.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-03T18:34:21.976 INFO:teuthology.orchestra.run.smithi032.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-03T18:34:21.976 INFO:teuthology.orchestra.run.smithi032.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-03T18:34:21.976 INFO:teuthology.orchestra.run.smithi032.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-05-03T18:34:21.976 INFO:teuthology.orchestra.run.smithi032.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-03T18:34:21.976 INFO:teuthology.orchestra.run.smithi032.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-03T18:34:21.977 INFO:teuthology.orchestra.run.smithi032.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-03T18:34:21.977 INFO:teuthology.orchestra.run.smithi032.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-03T18:34:21.977 INFO:teuthology.orchestra.run.smithi032.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-03T18:34:21.977 INFO:teuthology.orchestra.run.smithi032.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-03T18:34:21.977 INFO:teuthology.orchestra.run.smithi032.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-03T18:34:21.977 INFO:teuthology.orchestra.run.smithi032.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-03T18:34:21.977 INFO:teuthology.orchestra.run.smithi032.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-03T18:34:21.977 INFO:teuthology.orchestra.run.smithi032.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-03T18:34:21.977 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-03T18:34:21.977 INFO:teuthology.orchestra.run.smithi032.stdout: python3-ceph-argparse x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 45 k 2024-05-03T18:34:21.977 INFO:teuthology.orchestra.run.smithi032.stdout: python3-ceph-common x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 135 k 2024-05-03T18:34:21.977 INFO:teuthology.orchestra.run.smithi032.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-03T18:34:21.977 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-03T18:34:21.977 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-03T18:34:21.978 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-03T18:34:21.978 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-03T18:34:21.978 INFO:teuthology.orchestra.run.smithi032.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-05-03T18:34:21.978 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-03T18:34:21.978 INFO:teuthology.orchestra.run.smithi032.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-03T18:34:21.978 INFO:teuthology.orchestra.run.smithi032.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-03T18:34:21.978 INFO:teuthology.orchestra.run.smithi032.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-03T18:34:21.978 INFO:teuthology.orchestra.run.smithi032.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-03T18:34:21.978 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-03T18:34:21.978 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-03T18:34:21.978 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-03T18:34:21.978 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-03T18:34:21.978 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-03T18:34:21.978 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-03T18:34:21.979 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-03T18:34:21.979 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-03T18:34:21.979 INFO:teuthology.orchestra.run.smithi032.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-03T18:34:21.979 INFO:teuthology.orchestra.run.smithi032.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-03T18:34:21.979 INFO:teuthology.orchestra.run.smithi032.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-03T18:34:21.979 INFO:teuthology.orchestra.run.smithi032.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-03T18:34:21.979 INFO:teuthology.orchestra.run.smithi032.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-03T18:34:21.979 INFO:teuthology.orchestra.run.smithi032.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-03T18:34:21.979 INFO:teuthology.orchestra.run.smithi032.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-03T18:34:21.979 INFO:teuthology.orchestra.run.smithi032.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-03T18:34:21.979 INFO:teuthology.orchestra.run.smithi032.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-03T18:34:21.979 INFO:teuthology.orchestra.run.smithi032.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-03T18:34:21.979 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-03T18:34:21.979 INFO:teuthology.orchestra.run.smithi032.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-03T18:34:21.980 INFO:teuthology.orchestra.run.smithi032.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-03T18:34:21.980 INFO:teuthology.orchestra.run.smithi032.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-03T18:34:21.980 INFO:teuthology.orchestra.run.smithi032.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-03T18:34:21.980 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-03T18:34:21.980 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-03T18:34:21.980 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-03T18:34:21.980 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-03T18:34:21.980 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-03T18:34:21.980 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-03T18:34:21.980 INFO:teuthology.orchestra.run.smithi032.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-03T18:34:21.980 INFO:teuthology.orchestra.run.smithi032.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-03T18:34:21.980 INFO:teuthology.orchestra.run.smithi032.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-03T18:34:21.980 INFO:teuthology.orchestra.run.smithi032.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-03T18:34:21.980 INFO:teuthology.orchestra.run.smithi032.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-03T18:34:21.980 INFO:teuthology.orchestra.run.smithi032.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-03T18:34:21.981 INFO:teuthology.orchestra.run.smithi032.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-03T18:34:21.981 INFO:teuthology.orchestra.run.smithi032.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-03T18:34:21.981 INFO:teuthology.orchestra.run.smithi032.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-03T18:34:21.981 INFO:teuthology.orchestra.run.smithi032.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-03T18:34:21.981 INFO:teuthology.orchestra.run.smithi032.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-03T18:34:21.981 INFO:teuthology.orchestra.run.smithi032.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-03T18:34:21.981 INFO:teuthology.orchestra.run.smithi032.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-03T18:34:21.981 INFO:teuthology.orchestra.run.smithi032.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-03T18:34:21.981 INFO:teuthology.orchestra.run.smithi032.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-03T18:34:21.981 INFO:teuthology.orchestra.run.smithi032.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-03T18:34:21.981 INFO:teuthology.orchestra.run.smithi032.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-03T18:34:21.981 INFO:teuthology.orchestra.run.smithi032.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-03T18:34:21.981 INFO:teuthology.orchestra.run.smithi032.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-03T18:34:21.981 INFO:teuthology.orchestra.run.smithi032.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-03T18:34:21.981 INFO:teuthology.orchestra.run.smithi032.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-03T18:34:21.982 INFO:teuthology.orchestra.run.smithi032.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-03T18:34:21.982 INFO:teuthology.orchestra.run.smithi032.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-03T18:34:21.982 INFO:teuthology.orchestra.run.smithi032.stdout:Installing weak dependencies: 2024-05-03T18:34:21.982 INFO:teuthology.orchestra.run.smithi032.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-03T18:34:21.982 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-03T18:34:21.982 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction Summary 2024-05-03T18:34:21.982 INFO:teuthology.orchestra.run.smithi032.stdout:======================================================================================= 2024-05-03T18:34:21.982 INFO:teuthology.orchestra.run.smithi032.stdout:Install 148 Packages 2024-05-03T18:34:21.982 INFO:teuthology.orchestra.run.smithi032.stdout:Upgrade 4 Packages 2024-05-03T18:34:21.982 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-03T18:34:21.983 INFO:teuthology.orchestra.run.smithi032.stdout:Total download size: 222 M 2024-05-03T18:34:21.983 INFO:teuthology.orchestra.run.smithi032.stdout:Downloading Packages: 2024-05-03T18:34:22.164 INFO:teuthology.orchestra.run.smithi113.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-05-03T18:34:22.936 INFO:teuthology.orchestra.run.smithi032.stdout:(1/152): ceph-19.0.0-3450.g9a047fe7.el9.x86_64. 27 kB/s | 6.6 kB 00:00 2024-05-03T18:34:23.228 INFO:teuthology.orchestra.run.smithi032.stdout:(2/152): ceph-base-19.0.0-3450.g9a047fe7.el9.x8 10 MB/s | 5.4 MB 00:00 2024-05-03T18:34:23.278 INFO:teuthology.orchestra.run.smithi032.stdout:(3/152): ceph-immutable-object-cache-19.0.0-345 2.8 MB/s | 145 kB 00:00 2024-05-03T18:34:23.445 INFO:teuthology.orchestra.run.smithi032.stdout:(4/152): ceph-mds-19.0.0-3450.g9a047fe7.el9.x86 14 MB/s | 2.4 MB 00:00 2024-05-03T18:34:23.486 INFO:teuthology.orchestra.run.smithi032.stdout:(5/152): ceph-fuse-19.0.0-3450.g9a047fe7.el9.x8 1.6 MB/s | 891 kB 00:00 2024-05-03T18:34:23.570 INFO:teuthology.orchestra.run.smithi032.stdout:(6/152): ceph-mgr-19.0.0-3450.g9a047fe7.el9.x86 12 MB/s | 1.5 MB 00:00 2024-05-03T18:34:24.117 INFO:teuthology.orchestra.run.smithi045.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-03T18:34:24.117 INFO:teuthology.orchestra.run.smithi045.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-03T18:34:24.129 INFO:teuthology.orchestra.run.smithi045.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-03T18:34:24.129 INFO:teuthology.orchestra.run.smithi045.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-03T18:34:24.130 INFO:teuthology.orchestra.run.smithi045.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-03T18:34:24.131 INFO:teuthology.orchestra.run.smithi045.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-03T18:34:24.133 INFO:teuthology.orchestra.run.smithi045.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-03T18:34:24.135 INFO:teuthology.orchestra.run.smithi045.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-03T18:34:24.136 INFO:teuthology.orchestra.run.smithi045.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-03T18:34:24.200 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-05-03T18:34:24.223 INFO:teuthology.orchestra.run.smithi045.stdout:======================================================================================= 2024-05-03T18:34:24.223 INFO:teuthology.orchestra.run.smithi045.stdout: Package Arch Version Repository Size 2024-05-03T18:34:24.223 INFO:teuthology.orchestra.run.smithi045.stdout:======================================================================================= 2024-05-03T18:34:24.223 INFO:teuthology.orchestra.run.smithi045.stdout:Installing: 2024-05-03T18:34:24.223 INFO:teuthology.orchestra.run.smithi045.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-03T18:34:24.224 INFO:teuthology.orchestra.run.smithi045.stdout: ceph x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 6.6 k 2024-05-03T18:34:24.224 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-base x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 5.4 M 2024-05-03T18:34:24.224 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-fuse x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 891 k 2024-05-03T18:34:24.224 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 145 k 2024-05-03T18:34:24.224 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 1.5 M 2024-05-03T18:34:24.224 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3450.g9a047fe7.el9 ceph-noarch 142 k 2024-05-03T18:34:24.224 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3450.g9a047fe7.el9 ceph-noarch 3.5 M 2024-05-03T18:34:24.224 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3450.g9a047fe7.el9 ceph-noarch 7.4 M 2024-05-03T18:34:24.224 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-rook noarch 2:19.0.0-3450.g9a047fe7.el9 ceph-noarch 49 k 2024-05-03T18:34:24.224 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-radosgw x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 11 M 2024-05-03T18:34:24.224 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-test x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 49 M 2024-05-03T18:34:24.225 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-volume noarch 2:19.0.0-3450.g9a047fe7.el9 ceph-noarch 273 k 2024-05-03T18:34:24.225 INFO:teuthology.orchestra.run.smithi045.stdout: cephadm noarch 2:19.0.0-3450.g9a047fe7.el9 ceph-noarch 763 k 2024-05-03T18:34:24.225 INFO:teuthology.orchestra.run.smithi045.stdout: cephfs-mirror x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 225 k 2024-05-03T18:34:24.225 INFO:teuthology.orchestra.run.smithi045.stdout: cephfs-shell x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 27 k 2024-05-03T18:34:24.225 INFO:teuthology.orchestra.run.smithi045.stdout: cephfs-top noarch 2:19.0.0-3450.g9a047fe7.el9 ceph-noarch 18 k 2024-05-03T18:34:24.225 INFO:teuthology.orchestra.run.smithi045.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-05-03T18:34:24.225 INFO:teuthology.orchestra.run.smithi045.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-03T18:34:24.225 INFO:teuthology.orchestra.run.smithi045.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-03T18:34:24.225 INFO:teuthology.orchestra.run.smithi045.stdout: libcephfs-devel x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 32 k 2024-05-03T18:34:24.225 INFO:teuthology.orchestra.run.smithi045.stdout: libcephfs2 x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 747 k 2024-05-03T18:34:24.225 INFO:teuthology.orchestra.run.smithi045.stdout: librados-devel x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 127 k 2024-05-03T18:34:24.225 INFO:teuthology.orchestra.run.smithi045.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-03T18:34:24.225 INFO:teuthology.orchestra.run.smithi045.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-03T18:34:24.225 INFO:teuthology.orchestra.run.smithi045.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-03T18:34:24.226 INFO:teuthology.orchestra.run.smithi045.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-03T18:34:24.226 INFO:teuthology.orchestra.run.smithi045.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-03T18:34:24.226 INFO:teuthology.orchestra.run.smithi045.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-03T18:34:24.226 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cephfs x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 161 k 2024-05-03T18:34:24.226 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rados x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 320 k 2024-05-03T18:34:24.226 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rbd x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 298 k 2024-05-03T18:34:24.226 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rgw x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 100 k 2024-05-03T18:34:24.226 INFO:teuthology.orchestra.run.smithi045.stdout: rbd-fuse x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 85 k 2024-05-03T18:34:24.226 INFO:teuthology.orchestra.run.smithi045.stdout: rbd-mirror x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 3.1 M 2024-05-03T18:34:24.226 INFO:teuthology.orchestra.run.smithi045.stdout: rbd-nbd x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 170 k 2024-05-03T18:34:24.226 INFO:teuthology.orchestra.run.smithi045.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-03T18:34:24.226 INFO:teuthology.orchestra.run.smithi045.stdout:Upgrading: 2024-05-03T18:34:24.226 INFO:teuthology.orchestra.run.smithi045.stdout: librados2 x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 3.4 M 2024-05-03T18:34:24.227 INFO:teuthology.orchestra.run.smithi045.stdout: librbd1 x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 3.1 M 2024-05-03T18:34:24.227 INFO:teuthology.orchestra.run.smithi045.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-03T18:34:24.227 INFO:teuthology.orchestra.run.smithi045.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-03T18:34:24.227 INFO:teuthology.orchestra.run.smithi045.stdout:Installing dependencies: 2024-05-03T18:34:24.227 INFO:teuthology.orchestra.run.smithi045.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-03T18:34:24.227 INFO:teuthology.orchestra.run.smithi045.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-03T18:34:24.227 INFO:teuthology.orchestra.run.smithi045.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-03T18:34:24.227 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-common x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 21 M 2024-05-03T18:34:24.227 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3450.g9a047fe7.el9 ceph-noarch 34 k 2024-05-03T18:34:24.227 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mds x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 2.4 M 2024-05-03T18:34:24.227 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3450.g9a047fe7.el9 ceph-noarch 274 k 2024-05-03T18:34:24.227 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mon x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 4.7 M 2024-05-03T18:34:24.227 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-osd x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 17 M 2024-05-03T18:34:24.227 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3450.g9a047fe7.el9 ceph-noarch 16 k 2024-05-03T18:34:24.227 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-selinux x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 25 k 2024-05-03T18:34:24.228 INFO:teuthology.orchestra.run.smithi045.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-03T18:34:24.228 INFO:teuthology.orchestra.run.smithi045.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-03T18:34:24.228 INFO:teuthology.orchestra.run.smithi045.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-03T18:34:24.228 INFO:teuthology.orchestra.run.smithi045.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-03T18:34:24.228 INFO:teuthology.orchestra.run.smithi045.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-03T18:34:24.228 INFO:teuthology.orchestra.run.smithi045.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-03T18:34:24.228 INFO:teuthology.orchestra.run.smithi045.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-03T18:34:24.228 INFO:teuthology.orchestra.run.smithi045.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-03T18:34:24.228 INFO:teuthology.orchestra.run.smithi045.stdout: libcephsqlite x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 163 k 2024-05-03T18:34:24.228 INFO:teuthology.orchestra.run.smithi045.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-03T18:34:24.228 INFO:teuthology.orchestra.run.smithi045.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-03T18:34:24.228 INFO:teuthology.orchestra.run.smithi045.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-03T18:34:24.228 INFO:teuthology.orchestra.run.smithi045.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-03T18:34:24.228 INFO:teuthology.orchestra.run.smithi045.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-03T18:34:24.229 INFO:teuthology.orchestra.run.smithi045.stdout: libradosstriper1 x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 500 k 2024-05-03T18:34:24.229 INFO:teuthology.orchestra.run.smithi045.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-03T18:34:24.229 INFO:teuthology.orchestra.run.smithi045.stdout: librgw2 x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 5.4 M 2024-05-03T18:34:24.229 INFO:teuthology.orchestra.run.smithi045.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-03T18:34:24.229 INFO:teuthology.orchestra.run.smithi045.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-03T18:34:24.229 INFO:teuthology.orchestra.run.smithi045.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-03T18:34:24.229 INFO:teuthology.orchestra.run.smithi045.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-03T18:34:24.229 INFO:teuthology.orchestra.run.smithi045.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-03T18:34:24.229 INFO:teuthology.orchestra.run.smithi045.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-05-03T18:34:24.229 INFO:teuthology.orchestra.run.smithi045.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-03T18:34:24.229 INFO:teuthology.orchestra.run.smithi045.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-03T18:34:24.229 INFO:teuthology.orchestra.run.smithi045.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-03T18:34:24.230 INFO:teuthology.orchestra.run.smithi045.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-03T18:34:24.230 INFO:teuthology.orchestra.run.smithi045.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-03T18:34:24.230 INFO:teuthology.orchestra.run.smithi045.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-03T18:34:24.230 INFO:teuthology.orchestra.run.smithi045.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-03T18:34:24.230 INFO:teuthology.orchestra.run.smithi045.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-03T18:34:24.230 INFO:teuthology.orchestra.run.smithi045.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-03T18:34:24.230 INFO:teuthology.orchestra.run.smithi045.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-03T18:34:24.231 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-03T18:34:24.231 INFO:teuthology.orchestra.run.smithi045.stdout: python3-ceph-argparse x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 45 k 2024-05-03T18:34:24.231 INFO:teuthology.orchestra.run.smithi045.stdout: python3-ceph-common x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 135 k 2024-05-03T18:34:24.231 INFO:teuthology.orchestra.run.smithi045.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-03T18:34:24.231 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-03T18:34:24.231 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-03T18:34:24.231 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-03T18:34:24.231 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-03T18:34:24.232 INFO:teuthology.orchestra.run.smithi045.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-05-03T18:34:24.232 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-03T18:34:24.232 INFO:teuthology.orchestra.run.smithi045.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-03T18:34:24.232 INFO:teuthology.orchestra.run.smithi045.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-03T18:34:24.232 INFO:teuthology.orchestra.run.smithi045.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-03T18:34:24.232 INFO:teuthology.orchestra.run.smithi045.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-03T18:34:24.233 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-03T18:34:24.233 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-03T18:34:24.233 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-03T18:34:24.233 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-03T18:34:24.233 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-03T18:34:24.233 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-03T18:34:24.234 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-03T18:34:24.234 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-03T18:34:24.234 INFO:teuthology.orchestra.run.smithi045.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-03T18:34:24.234 INFO:teuthology.orchestra.run.smithi045.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-03T18:34:24.234 INFO:teuthology.orchestra.run.smithi045.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-03T18:34:24.234 INFO:teuthology.orchestra.run.smithi045.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-03T18:34:24.234 INFO:teuthology.orchestra.run.smithi045.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-03T18:34:24.235 INFO:teuthology.orchestra.run.smithi045.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-03T18:34:24.235 INFO:teuthology.orchestra.run.smithi045.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-03T18:34:24.235 INFO:teuthology.orchestra.run.smithi045.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-03T18:34:24.235 INFO:teuthology.orchestra.run.smithi045.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-03T18:34:24.235 INFO:teuthology.orchestra.run.smithi045.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-03T18:34:24.235 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-03T18:34:24.235 INFO:teuthology.orchestra.run.smithi045.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-03T18:34:24.235 INFO:teuthology.orchestra.run.smithi045.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-03T18:34:24.236 INFO:teuthology.orchestra.run.smithi045.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-03T18:34:24.236 INFO:teuthology.orchestra.run.smithi045.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-03T18:34:24.236 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-03T18:34:24.236 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-03T18:34:24.236 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-03T18:34:24.236 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-03T18:34:24.236 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-03T18:34:24.236 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-03T18:34:24.236 INFO:teuthology.orchestra.run.smithi045.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-03T18:34:24.237 INFO:teuthology.orchestra.run.smithi045.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-03T18:34:24.237 INFO:teuthology.orchestra.run.smithi045.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-03T18:34:24.237 INFO:teuthology.orchestra.run.smithi045.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-03T18:34:24.237 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-03T18:34:24.237 INFO:teuthology.orchestra.run.smithi045.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-03T18:34:24.237 INFO:teuthology.orchestra.run.smithi045.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-03T18:34:24.238 INFO:teuthology.orchestra.run.smithi045.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-03T18:34:24.238 INFO:teuthology.orchestra.run.smithi045.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-03T18:34:24.238 INFO:teuthology.orchestra.run.smithi045.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-03T18:34:24.238 INFO:teuthology.orchestra.run.smithi045.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-03T18:34:24.238 INFO:teuthology.orchestra.run.smithi045.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-03T18:34:24.238 INFO:teuthology.orchestra.run.smithi045.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-03T18:34:24.239 INFO:teuthology.orchestra.run.smithi045.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-03T18:34:24.239 INFO:teuthology.orchestra.run.smithi045.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-03T18:34:24.239 INFO:teuthology.orchestra.run.smithi045.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-03T18:34:24.239 INFO:teuthology.orchestra.run.smithi045.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-03T18:34:24.239 INFO:teuthology.orchestra.run.smithi045.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-03T18:34:24.239 INFO:teuthology.orchestra.run.smithi045.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-03T18:34:24.239 INFO:teuthology.orchestra.run.smithi045.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-03T18:34:24.240 INFO:teuthology.orchestra.run.smithi045.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-03T18:34:24.240 INFO:teuthology.orchestra.run.smithi045.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-03T18:34:24.240 INFO:teuthology.orchestra.run.smithi045.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-03T18:34:24.240 INFO:teuthology.orchestra.run.smithi045.stdout:Installing weak dependencies: 2024-05-03T18:34:24.240 INFO:teuthology.orchestra.run.smithi045.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-03T18:34:24.240 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-03T18:34:24.240 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2024-05-03T18:34:24.240 INFO:teuthology.orchestra.run.smithi045.stdout:======================================================================================= 2024-05-03T18:34:24.241 INFO:teuthology.orchestra.run.smithi045.stdout:Install 148 Packages 2024-05-03T18:34:24.241 INFO:teuthology.orchestra.run.smithi045.stdout:Upgrade 4 Packages 2024-05-03T18:34:24.241 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-03T18:34:24.241 INFO:teuthology.orchestra.run.smithi045.stdout:Total download size: 222 M 2024-05-03T18:34:24.241 INFO:teuthology.orchestra.run.smithi045.stdout:Downloading Packages: 2024-05-03T18:34:24.273 INFO:teuthology.orchestra.run.smithi113.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-03T18:34:24.273 INFO:teuthology.orchestra.run.smithi113.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-03T18:34:24.284 INFO:teuthology.orchestra.run.smithi113.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-03T18:34:24.285 INFO:teuthology.orchestra.run.smithi113.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-03T18:34:24.285 INFO:teuthology.orchestra.run.smithi113.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-03T18:34:24.286 INFO:teuthology.orchestra.run.smithi113.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-03T18:34:24.288 INFO:teuthology.orchestra.run.smithi113.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-03T18:34:24.290 INFO:teuthology.orchestra.run.smithi113.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-03T18:34:24.291 INFO:teuthology.orchestra.run.smithi113.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-03T18:34:24.351 INFO:teuthology.orchestra.run.smithi113.stdout:Dependencies resolved. 2024-05-03T18:34:24.366 INFO:teuthology.orchestra.run.smithi113.stdout:======================================================================================= 2024-05-03T18:34:24.366 INFO:teuthology.orchestra.run.smithi113.stdout: Package Arch Version Repository Size 2024-05-03T18:34:24.366 INFO:teuthology.orchestra.run.smithi113.stdout:======================================================================================= 2024-05-03T18:34:24.366 INFO:teuthology.orchestra.run.smithi113.stdout:Installing: 2024-05-03T18:34:24.366 INFO:teuthology.orchestra.run.smithi113.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-03T18:34:24.366 INFO:teuthology.orchestra.run.smithi113.stdout: ceph x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 6.6 k 2024-05-03T18:34:24.366 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-base x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 5.4 M 2024-05-03T18:34:24.366 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-fuse x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 891 k 2024-05-03T18:34:24.366 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 145 k 2024-05-03T18:34:24.366 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mgr x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 1.5 M 2024-05-03T18:34:24.366 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3450.g9a047fe7.el9 ceph-noarch 142 k 2024-05-03T18:34:24.367 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3450.g9a047fe7.el9 ceph-noarch 3.5 M 2024-05-03T18:34:24.367 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3450.g9a047fe7.el9 ceph-noarch 7.4 M 2024-05-03T18:34:24.367 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mgr-rook noarch 2:19.0.0-3450.g9a047fe7.el9 ceph-noarch 49 k 2024-05-03T18:34:24.367 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-radosgw x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 11 M 2024-05-03T18:34:24.367 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-test x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 49 M 2024-05-03T18:34:24.367 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-volume noarch 2:19.0.0-3450.g9a047fe7.el9 ceph-noarch 273 k 2024-05-03T18:34:24.367 INFO:teuthology.orchestra.run.smithi113.stdout: cephadm noarch 2:19.0.0-3450.g9a047fe7.el9 ceph-noarch 763 k 2024-05-03T18:34:24.367 INFO:teuthology.orchestra.run.smithi113.stdout: cephfs-mirror x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 225 k 2024-05-03T18:34:24.367 INFO:teuthology.orchestra.run.smithi113.stdout: cephfs-shell x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 27 k 2024-05-03T18:34:24.367 INFO:teuthology.orchestra.run.smithi113.stdout: cephfs-top noarch 2:19.0.0-3450.g9a047fe7.el9 ceph-noarch 18 k 2024-05-03T18:34:24.367 INFO:teuthology.orchestra.run.smithi113.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-05-03T18:34:24.367 INFO:teuthology.orchestra.run.smithi113.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-03T18:34:24.367 INFO:teuthology.orchestra.run.smithi113.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-03T18:34:24.367 INFO:teuthology.orchestra.run.smithi113.stdout: libcephfs-devel x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 32 k 2024-05-03T18:34:24.368 INFO:teuthology.orchestra.run.smithi113.stdout: libcephfs2 x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 747 k 2024-05-03T18:34:24.368 INFO:teuthology.orchestra.run.smithi113.stdout: librados-devel x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 127 k 2024-05-03T18:34:24.368 INFO:teuthology.orchestra.run.smithi113.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-03T18:34:24.368 INFO:teuthology.orchestra.run.smithi113.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-03T18:34:24.368 INFO:teuthology.orchestra.run.smithi113.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-03T18:34:24.368 INFO:teuthology.orchestra.run.smithi113.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-03T18:34:24.368 INFO:teuthology.orchestra.run.smithi113.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-03T18:34:24.368 INFO:teuthology.orchestra.run.smithi113.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-03T18:34:24.368 INFO:teuthology.orchestra.run.smithi113.stdout: python3-cephfs x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 161 k 2024-05-03T18:34:24.368 INFO:teuthology.orchestra.run.smithi113.stdout: python3-rados x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 320 k 2024-05-03T18:34:24.368 INFO:teuthology.orchestra.run.smithi113.stdout: python3-rbd x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 298 k 2024-05-03T18:34:24.368 INFO:teuthology.orchestra.run.smithi113.stdout: python3-rgw x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 100 k 2024-05-03T18:34:24.368 INFO:teuthology.orchestra.run.smithi113.stdout: rbd-fuse x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 85 k 2024-05-03T18:34:24.368 INFO:teuthology.orchestra.run.smithi113.stdout: rbd-mirror x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 3.1 M 2024-05-03T18:34:24.369 INFO:teuthology.orchestra.run.smithi113.stdout: rbd-nbd x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 170 k 2024-05-03T18:34:24.369 INFO:teuthology.orchestra.run.smithi113.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-03T18:34:24.369 INFO:teuthology.orchestra.run.smithi113.stdout:Upgrading: 2024-05-03T18:34:24.369 INFO:teuthology.orchestra.run.smithi113.stdout: librados2 x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 3.4 M 2024-05-03T18:34:24.369 INFO:teuthology.orchestra.run.smithi113.stdout: librbd1 x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 3.1 M 2024-05-03T18:34:24.369 INFO:teuthology.orchestra.run.smithi113.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-03T18:34:24.369 INFO:teuthology.orchestra.run.smithi113.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-03T18:34:24.369 INFO:teuthology.orchestra.run.smithi113.stdout:Installing dependencies: 2024-05-03T18:34:24.369 INFO:teuthology.orchestra.run.smithi113.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-03T18:34:24.369 INFO:teuthology.orchestra.run.smithi113.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-03T18:34:24.369 INFO:teuthology.orchestra.run.smithi113.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-03T18:34:24.369 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-common x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 21 M 2024-05-03T18:34:24.369 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3450.g9a047fe7.el9 ceph-noarch 34 k 2024-05-03T18:34:24.369 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mds x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 2.4 M 2024-05-03T18:34:24.370 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3450.g9a047fe7.el9 ceph-noarch 274 k 2024-05-03T18:34:24.370 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mon x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 4.7 M 2024-05-03T18:34:24.370 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-osd x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 17 M 2024-05-03T18:34:24.370 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3450.g9a047fe7.el9 ceph-noarch 16 k 2024-05-03T18:34:24.370 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-selinux x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 25 k 2024-05-03T18:34:24.370 INFO:teuthology.orchestra.run.smithi113.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-03T18:34:24.370 INFO:teuthology.orchestra.run.smithi113.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-03T18:34:24.370 INFO:teuthology.orchestra.run.smithi113.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-03T18:34:24.370 INFO:teuthology.orchestra.run.smithi113.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-03T18:34:24.370 INFO:teuthology.orchestra.run.smithi113.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-03T18:34:24.370 INFO:teuthology.orchestra.run.smithi113.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-03T18:34:24.370 INFO:teuthology.orchestra.run.smithi113.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-03T18:34:24.370 INFO:teuthology.orchestra.run.smithi113.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-03T18:34:24.370 INFO:teuthology.orchestra.run.smithi113.stdout: libcephsqlite x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 163 k 2024-05-03T18:34:24.370 INFO:teuthology.orchestra.run.smithi113.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-03T18:34:24.371 INFO:teuthology.orchestra.run.smithi113.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-03T18:34:24.371 INFO:teuthology.orchestra.run.smithi113.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-03T18:34:24.371 INFO:teuthology.orchestra.run.smithi113.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-03T18:34:24.371 INFO:teuthology.orchestra.run.smithi113.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-03T18:34:24.371 INFO:teuthology.orchestra.run.smithi113.stdout: libradosstriper1 x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 500 k 2024-05-03T18:34:24.371 INFO:teuthology.orchestra.run.smithi113.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-03T18:34:24.371 INFO:teuthology.orchestra.run.smithi113.stdout: librgw2 x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 5.4 M 2024-05-03T18:34:24.371 INFO:teuthology.orchestra.run.smithi113.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-03T18:34:24.371 INFO:teuthology.orchestra.run.smithi113.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-03T18:34:24.371 INFO:teuthology.orchestra.run.smithi113.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-03T18:34:24.371 INFO:teuthology.orchestra.run.smithi113.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-03T18:34:24.371 INFO:teuthology.orchestra.run.smithi113.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-03T18:34:24.371 INFO:teuthology.orchestra.run.smithi113.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-05-03T18:34:24.371 INFO:teuthology.orchestra.run.smithi113.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-03T18:34:24.372 INFO:teuthology.orchestra.run.smithi113.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-03T18:34:24.372 INFO:teuthology.orchestra.run.smithi113.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-03T18:34:24.372 INFO:teuthology.orchestra.run.smithi113.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-03T18:34:24.372 INFO:teuthology.orchestra.run.smithi113.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-03T18:34:24.372 INFO:teuthology.orchestra.run.smithi113.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-03T18:34:24.372 INFO:teuthology.orchestra.run.smithi113.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-03T18:34:24.372 INFO:teuthology.orchestra.run.smithi113.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-03T18:34:24.372 INFO:teuthology.orchestra.run.smithi113.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-03T18:34:24.372 INFO:teuthology.orchestra.run.smithi113.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-03T18:34:24.372 INFO:teuthology.orchestra.run.smithi113.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-03T18:34:24.372 INFO:teuthology.orchestra.run.smithi113.stdout: python3-ceph-argparse x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 45 k 2024-05-03T18:34:24.372 INFO:teuthology.orchestra.run.smithi113.stdout: python3-ceph-common x86_64 2:19.0.0-3450.g9a047fe7.el9 ceph 135 k 2024-05-03T18:34:24.372 INFO:teuthology.orchestra.run.smithi113.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-03T18:34:24.372 INFO:teuthology.orchestra.run.smithi113.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-03T18:34:24.372 INFO:teuthology.orchestra.run.smithi113.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-03T18:34:24.373 INFO:teuthology.orchestra.run.smithi113.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-03T18:34:24.373 INFO:teuthology.orchestra.run.smithi113.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-03T18:34:24.373 INFO:teuthology.orchestra.run.smithi113.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-05-03T18:34:24.373 INFO:teuthology.orchestra.run.smithi113.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-03T18:34:24.373 INFO:teuthology.orchestra.run.smithi113.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-03T18:34:24.373 INFO:teuthology.orchestra.run.smithi113.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-03T18:34:24.373 INFO:teuthology.orchestra.run.smithi113.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-03T18:34:24.373 INFO:teuthology.orchestra.run.smithi113.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-03T18:34:24.373 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-03T18:34:24.373 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-03T18:34:24.373 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-03T18:34:24.373 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-03T18:34:24.373 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-03T18:34:24.374 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-03T18:34:24.374 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-03T18:34:24.374 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-03T18:34:24.374 INFO:teuthology.orchestra.run.smithi113.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-03T18:34:24.374 INFO:teuthology.orchestra.run.smithi113.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-03T18:34:24.374 INFO:teuthology.orchestra.run.smithi113.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-03T18:34:24.374 INFO:teuthology.orchestra.run.smithi113.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-03T18:34:24.374 INFO:teuthology.orchestra.run.smithi113.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-03T18:34:24.374 INFO:teuthology.orchestra.run.smithi113.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-03T18:34:24.374 INFO:teuthology.orchestra.run.smithi113.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-03T18:34:24.374 INFO:teuthology.orchestra.run.smithi113.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-03T18:34:24.374 INFO:teuthology.orchestra.run.smithi113.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-03T18:34:24.374 INFO:teuthology.orchestra.run.smithi113.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-03T18:34:24.374 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-03T18:34:24.375 INFO:teuthology.orchestra.run.smithi113.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-03T18:34:24.375 INFO:teuthology.orchestra.run.smithi113.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-03T18:34:24.375 INFO:teuthology.orchestra.run.smithi113.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-03T18:34:24.375 INFO:teuthology.orchestra.run.smithi113.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-03T18:34:24.375 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-03T18:34:24.375 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-03T18:34:24.375 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-03T18:34:24.375 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-03T18:34:24.375 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-03T18:34:24.375 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-03T18:34:24.375 INFO:teuthology.orchestra.run.smithi113.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-03T18:34:24.375 INFO:teuthology.orchestra.run.smithi113.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-03T18:34:24.375 INFO:teuthology.orchestra.run.smithi113.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-03T18:34:24.376 INFO:teuthology.orchestra.run.smithi113.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-03T18:34:24.376 INFO:teuthology.orchestra.run.smithi113.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-03T18:34:24.376 INFO:teuthology.orchestra.run.smithi113.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-03T18:34:24.376 INFO:teuthology.orchestra.run.smithi113.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-03T18:34:24.376 INFO:teuthology.orchestra.run.smithi113.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-03T18:34:24.376 INFO:teuthology.orchestra.run.smithi113.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-03T18:34:24.376 INFO:teuthology.orchestra.run.smithi113.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-03T18:34:24.376 INFO:teuthology.orchestra.run.smithi113.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-03T18:34:24.376 INFO:teuthology.orchestra.run.smithi113.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-03T18:34:24.376 INFO:teuthology.orchestra.run.smithi113.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-03T18:34:24.376 INFO:teuthology.orchestra.run.smithi113.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-03T18:34:24.376 INFO:teuthology.orchestra.run.smithi113.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-03T18:34:24.376 INFO:teuthology.orchestra.run.smithi113.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-03T18:34:24.376 INFO:teuthology.orchestra.run.smithi113.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-03T18:34:24.377 INFO:teuthology.orchestra.run.smithi113.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-03T18:34:24.377 INFO:teuthology.orchestra.run.smithi113.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-03T18:34:24.377 INFO:teuthology.orchestra.run.smithi113.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-03T18:34:24.377 INFO:teuthology.orchestra.run.smithi113.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-03T18:34:24.377 INFO:teuthology.orchestra.run.smithi113.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-03T18:34:24.377 INFO:teuthology.orchestra.run.smithi113.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-03T18:34:24.377 INFO:teuthology.orchestra.run.smithi113.stdout:Installing weak dependencies: 2024-05-03T18:34:24.377 INFO:teuthology.orchestra.run.smithi113.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-03T18:34:24.377 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-05-03T18:34:24.377 INFO:teuthology.orchestra.run.smithi113.stdout:Transaction Summary 2024-05-03T18:34:24.377 INFO:teuthology.orchestra.run.smithi113.stdout:======================================================================================= 2024-05-03T18:34:24.377 INFO:teuthology.orchestra.run.smithi113.stdout:Install 148 Packages 2024-05-03T18:34:24.377 INFO:teuthology.orchestra.run.smithi113.stdout:Upgrade 4 Packages 2024-05-03T18:34:24.377 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-05-03T18:34:24.378 INFO:teuthology.orchestra.run.smithi113.stdout:Total download size: 222 M 2024-05-03T18:34:24.378 INFO:teuthology.orchestra.run.smithi113.stdout:Downloading Packages: 2024-05-03T18:34:24.529 INFO:teuthology.orchestra.run.smithi032.stdout:(7/152): ceph-osd-19.0.0-3450.g9a047fe7.el9.x86 18 MB/s | 17 MB 00:00 2024-05-03T18:34:25.107 INFO:teuthology.orchestra.run.smithi113.stdout:(1/152): ceph-19.0.0-3450.g9a047fe7.el9.x86_64. 17 kB/s | 6.6 kB 00:00 2024-05-03T18:34:25.137 INFO:teuthology.orchestra.run.smithi032.stdout:(8/152): ceph-mon-19.0.0-3450.g9a047fe7.el9.x86 2.8 MB/s | 4.7 MB 00:01 2024-05-03T18:34:25.254 INFO:teuthology.orchestra.run.smithi032.stdout:(9/152): ceph-radosgw-19.0.0-3450.g9a047fe7.el9 15 MB/s | 11 MB 00:00 2024-05-03T18:34:25.280 INFO:teuthology.orchestra.run.smithi032.stdout:(10/152): ceph-selinux-19.0.0-3450.g9a047fe7.el 177 kB/s | 25 kB 00:00 2024-05-03T18:34:25.299 INFO:teuthology.orchestra.run.smithi045.stdout:(1/152): ceph-19.0.0-3450.g9a047fe7.el9.x86_64. 11 kB/s | 6.6 kB 00:00 2024-05-03T18:34:25.606 INFO:teuthology.orchestra.run.smithi113.stdout:(2/152): ceph-fuse-19.0.0-3450.g9a047fe7.el9.x8 1.7 MB/s | 891 kB 00:00 2024-05-03T18:34:25.715 INFO:teuthology.orchestra.run.smithi113.stdout:(3/152): ceph-immutable-object-cache-19.0.0-345 1.3 MB/s | 145 kB 00:00 2024-05-03T18:34:26.022 INFO:teuthology.orchestra.run.smithi032.stdout:(11/152): cephfs-mirror-19.0.0-3450.g9a047fe7.e 303 kB/s | 225 kB 00:00 2024-05-03T18:34:26.147 INFO:teuthology.orchestra.run.smithi032.stdout:(12/152): cephfs-shell-19.0.0-3450.g9a047fe7.el 217 kB/s | 27 kB 00:00 2024-05-03T18:34:26.247 INFO:teuthology.orchestra.run.smithi032.stdout:(13/152): libcephfs-devel-19.0.0-3450.g9a047fe7 317 kB/s | 32 kB 00:00 2024-05-03T18:34:26.324 INFO:teuthology.orchestra.run.smithi045.stdout:(2/152): ceph-base-19.0.0-3450.g9a047fe7.el9.x8 3.3 MB/s | 5.4 MB 00:01 2024-05-03T18:34:26.424 INFO:teuthology.orchestra.run.smithi045.stdout:(3/152): ceph-immutable-object-cache-19.0.0-345 1.4 MB/s | 145 kB 00:00 2024-05-03T18:34:26.507 INFO:teuthology.orchestra.run.smithi113.stdout:(4/152): ceph-base-19.0.0-3450.g9a047fe7.el9.x8 3.0 MB/s | 5.4 MB 00:01 2024-05-03T18:34:26.531 INFO:teuthology.orchestra.run.smithi032.stdout:(14/152): libcephfs2-19.0.0-3450.g9a047fe7.el9. 2.6 MB/s | 747 kB 00:00 2024-05-03T18:34:26.691 INFO:teuthology.orchestra.run.smithi113.stdout:(5/152): ceph-mds-19.0.0-3450.g9a047fe7.el9.x86 2.4 MB/s | 2.4 MB 00:00 2024-05-03T18:34:26.882 INFO:teuthology.orchestra.run.smithi113.stdout:(6/152): ceph-mgr-19.0.0-3450.g9a047fe7.el9.x86 4.0 MB/s | 1.5 MB 00:00 2024-05-03T18:34:26.933 INFO:teuthology.orchestra.run.smithi045.stdout:(4/152): ceph-mds-19.0.0-3450.g9a047fe7.el9.x86 4.7 MB/s | 2.4 MB 00:00 2024-05-03T18:34:27.308 INFO:teuthology.orchestra.run.smithi045.stdout:(5/152): ceph-mgr-19.0.0-3450.g9a047fe7.el9.x86 4.0 MB/s | 1.5 MB 00:00 2024-05-03T18:34:27.448 INFO:teuthology.orchestra.run.smithi032.stdout:(15/152): libcephsqlite-19.0.0-3450.g9a047fe7.e 178 kB/s | 163 kB 00:00 2024-05-03T18:34:27.557 INFO:teuthology.orchestra.run.smithi032.stdout:(16/152): librados-devel-19.0.0-3450.g9a047fe7. 1.1 MB/s | 127 kB 00:00 2024-05-03T18:34:27.857 INFO:teuthology.orchestra.run.smithi032.stdout:(17/152): libradosstriper1-19.0.0-3450.g9a047fe 1.6 MB/s | 500 kB 00:00 2024-05-03T18:34:28.184 INFO:teuthology.orchestra.run.smithi045.stdout:(6/152): ceph-fuse-19.0.0-3450.g9a047fe7.el9.x8 309 kB/s | 891 kB 00:02 2024-05-03T18:34:28.275 INFO:teuthology.orchestra.run.smithi045.stdout:(7/152): ceph-mon-19.0.0-3450.g9a047fe7.el9.x86 4.9 MB/s | 4.7 MB 00:00 2024-05-03T18:34:28.582 INFO:teuthology.orchestra.run.smithi032.stdout:(18/152): ceph-common-19.0.0-3450.g9a047fe7.el9 3.6 MB/s | 21 MB 00:05 2024-05-03T18:34:28.633 INFO:teuthology.orchestra.run.smithi113.stdout:(7/152): ceph-mon-19.0.0-3450.g9a047fe7.el9.x86 2.4 MB/s | 4.7 MB 00:01 2024-05-03T18:34:28.674 INFO:teuthology.orchestra.run.smithi032.stdout:(19/152): python3-ceph-argparse-19.0.0-3450.g9a 494 kB/s | 45 kB 00:00 2024-05-03T18:34:28.791 INFO:teuthology.orchestra.run.smithi032.stdout:(20/152): python3-ceph-common-19.0.0-3450.g9a04 1.1 MB/s | 135 kB 00:00 2024-05-03T18:34:28.900 INFO:teuthology.orchestra.run.smithi032.stdout:(21/152): python3-cephfs-19.0.0-3450.g9a047fe7. 1.5 MB/s | 161 kB 00:00 2024-05-03T18:34:29.091 INFO:teuthology.orchestra.run.smithi032.stdout:(22/152): python3-rados-19.0.0-3450.g9a047fe7.e 1.6 MB/s | 320 kB 00:00 2024-05-03T18:34:29.300 INFO:teuthology.orchestra.run.smithi032.stdout:(23/152): python3-rbd-19.0.0-3450.g9a047fe7.el9 1.4 MB/s | 298 kB 00:00 2024-05-03T18:34:29.393 INFO:teuthology.orchestra.run.smithi032.stdout:(24/152): python3-rgw-19.0.0-3450.g9a047fe7.el9 1.1 MB/s | 100 kB 00:00 2024-05-03T18:34:29.482 INFO:teuthology.orchestra.run.smithi113.stdout:(8/152): ceph-common-19.0.0-3450.g9a047fe7.el9. 4.4 MB/s | 21 MB 00:04 2024-05-03T18:34:29.492 INFO:teuthology.orchestra.run.smithi032.stdout:(25/152): rbd-fuse-19.0.0-3450.g9a047fe7.el9.x8 855 kB/s | 85 kB 00:00 2024-05-03T18:34:29.575 INFO:teuthology.orchestra.run.smithi113.stdout:(9/152): ceph-selinux-19.0.0-3450.g9a047fe7.el9 275 kB/s | 25 kB 00:00 2024-05-03T18:34:30.167 INFO:teuthology.orchestra.run.smithi045.stdout:(8/152): ceph-radosgw-19.0.0-3450.g9a047fe7.el9 5.6 MB/s | 11 MB 00:01 2024-05-03T18:34:30.184 INFO:teuthology.orchestra.run.smithi032.stdout:(26/152): librgw2-19.0.0-3450.g9a047fe7.el9.x86 2.3 MB/s | 5.4 MB 00:02 2024-05-03T18:34:30.259 INFO:teuthology.orchestra.run.smithi045.stdout:(9/152): ceph-selinux-19.0.0-3450.g9a047fe7.el9 275 kB/s | 25 kB 00:00 2024-05-03T18:34:30.276 INFO:teuthology.orchestra.run.smithi032.stdout:(27/152): rbd-nbd-19.0.0-3450.g9a047fe7.el9.x86 1.8 MB/s | 170 kB 00:00 2024-05-03T18:34:30.366 INFO:teuthology.orchestra.run.smithi113.stdout:(10/152): ceph-osd-19.0.0-3450.g9a047fe7.el9.x8 4.8 MB/s | 17 MB 00:03 2024-05-03T18:34:30.402 INFO:teuthology.orchestra.run.smithi032.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3450.g 273 kB/s | 34 kB 00:00 2024-05-03T18:34:30.466 INFO:teuthology.orchestra.run.smithi113.stdout:(11/152): cephfs-mirror-19.0.0-3450.g9a047fe7.e 2.2 MB/s | 225 kB 00:00 2024-05-03T18:34:30.510 INFO:teuthology.orchestra.run.smithi032.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3450.g9a047fe 1.3 MB/s | 142 kB 00:00 2024-05-03T18:34:30.583 INFO:teuthology.orchestra.run.smithi113.stdout:(12/152): cephfs-shell-19.0.0-3450.g9a047fe7.el 231 kB/s | 27 kB 00:00 2024-05-03T18:34:30.684 INFO:teuthology.orchestra.run.smithi113.stdout:(13/152): libcephfs-devel-19.0.0-3450.g9a047fe7 317 kB/s | 32 kB 00:00 2024-05-03T18:34:30.844 INFO:teuthology.orchestra.run.smithi032.stdout:(30/152): rbd-mirror-19.0.0-3450.g9a047fe7.el9. 2.3 MB/s | 3.1 MB 00:01 2024-05-03T18:34:30.901 INFO:teuthology.orchestra.run.smithi113.stdout:(14/152): libcephfs2-19.0.0-3450.g9a047fe7.el9. 3.4 MB/s | 747 kB 00:00 2024-05-03T18:34:31.018 INFO:teuthology.orchestra.run.smithi113.stdout:(15/152): libcephsqlite-19.0.0-3450.g9a047fe7.e 1.4 MB/s | 163 kB 00:00 2024-05-03T18:34:31.126 INFO:teuthology.orchestra.run.smithi113.stdout:(16/152): librados-devel-19.0.0-3450.g9a047fe7. 1.1 MB/s | 127 kB 00:00 2024-05-03T18:34:31.335 INFO:teuthology.orchestra.run.smithi113.stdout:(17/152): libradosstriper1-19.0.0-3450.g9a047fe 2.3 MB/s | 500 kB 00:00 2024-05-03T18:34:31.869 INFO:teuthology.orchestra.run.smithi032.stdout:(31/152): ceph-mgr-dashboard-19.0.0-3450.g9a047 2.6 MB/s | 3.5 MB 00:01 2024-05-03T18:34:32.070 INFO:teuthology.orchestra.run.smithi032.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3450.g9a 1.3 MB/s | 274 kB 00:00 2024-05-03T18:34:32.178 INFO:teuthology.orchestra.run.smithi032.stdout:(33/152): ceph-mgr-rook-19.0.0-3450.g9a047fe7.e 456 kB/s | 49 kB 00:00 2024-05-03T18:34:32.278 INFO:teuthology.orchestra.run.smithi032.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3450.g9 165 kB/s | 16 kB 00:00 2024-05-03T18:34:32.402 INFO:teuthology.orchestra.run.smithi113.stdout:(18/152): librgw2-19.0.0-3450.g9a047fe7.el9.x86 5.0 MB/s | 5.4 MB 00:01 2024-05-03T18:34:32.445 INFO:teuthology.orchestra.run.smithi032.stdout:(35/152): ceph-volume-19.0.0-3450.g9a047fe7.el9 1.6 MB/s | 273 kB 00:00 2024-05-03T18:34:32.493 INFO:teuthology.orchestra.run.smithi113.stdout:(19/152): python3-ceph-argparse-19.0.0-3450.g9a 493 kB/s | 45 kB 00:00 2024-05-03T18:34:32.594 INFO:teuthology.orchestra.run.smithi113.stdout:(20/152): python3-ceph-common-19.0.0-3450.g9a04 1.3 MB/s | 135 kB 00:00 2024-05-03T18:34:32.736 INFO:teuthology.orchestra.run.smithi113.stdout:(21/152): ceph-radosgw-19.0.0-3450.g9a047fe7.el 2.6 MB/s | 11 MB 00:04 2024-05-03T18:34:32.762 INFO:teuthology.orchestra.run.smithi113.stdout:(22/152): python3-cephfs-19.0.0-3450.g9a047fe7. 968 kB/s | 161 kB 00:00 2024-05-03T18:34:32.837 INFO:teuthology.orchestra.run.smithi113.stdout:(23/152): python3-rbd-19.0.0-3450.g9a047fe7.el9 3.9 MB/s | 298 kB 00:00 2024-05-03T18:34:32.862 INFO:teuthology.orchestra.run.smithi113.stdout:(24/152): python3-rados-19.0.0-3450.g9a047fe7.e 2.5 MB/s | 320 kB 00:00 2024-05-03T18:34:32.912 INFO:teuthology.orchestra.run.smithi113.stdout:(25/152): python3-rgw-19.0.0-3450.g9a047fe7.el9 1.3 MB/s | 100 kB 00:00 2024-05-03T18:34:32.929 INFO:teuthology.orchestra.run.smithi113.stdout:(26/152): rbd-fuse-19.0.0-3450.g9a047fe7.el9.x8 1.2 MB/s | 85 kB 00:00 2024-05-03T18:34:33.021 INFO:teuthology.orchestra.run.smithi113.stdout:(27/152): rbd-nbd-19.0.0-3450.g9a047fe7.el9.x86 1.8 MB/s | 170 kB 00:00 2024-05-03T18:34:33.105 INFO:teuthology.orchestra.run.smithi113.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3450.g 410 kB/s | 34 kB 00:00 2024-05-03T18:34:33.129 INFO:teuthology.orchestra.run.smithi032.stdout:(36/152): ceph-test-19.0.0-3450.g9a047fe7.el9.x 6.2 MB/s | 49 MB 00:07 2024-05-03T18:34:33.154 INFO:teuthology.orchestra.run.smithi032.stdout:(37/152): cephadm-19.0.0-3450.g9a047fe7.el9.noa 1.1 MB/s | 763 kB 00:00 2024-05-03T18:34:33.189 INFO:teuthology.orchestra.run.smithi113.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3450.g9a047fe 1.7 MB/s | 142 kB 00:00 2024-05-03T18:34:33.196 INFO:teuthology.orchestra.run.smithi032.stdout:(38/152): cephfs-top-19.0.0-3450.g9a047fe7.el9. 273 kB/s | 18 kB 00:00 2024-05-03T18:34:33.372 INFO:teuthology.orchestra.run.smithi032.stdout:(39/152): fuse-2.9.9-15.el9.x86_64.rpm 370 kB/s | 80 kB 00:00 2024-05-03T18:34:33.405 INFO:teuthology.orchestra.run.smithi113.stdout:(30/152): rbd-mirror-19.0.0-3450.g9a047fe7.el9. 6.3 MB/s | 3.1 MB 00:00 2024-05-03T18:34:33.456 INFO:teuthology.orchestra.run.smithi032.stdout:(40/152): libquadmath-11.4.1-3.el9.x86_64.rpm 2.3 MB/s | 195 kB 00:00 2024-05-03T18:34:33.489 INFO:teuthology.orchestra.run.smithi032.stdout:(41/152): libgfortran-11.4.1-3.el9.x86_64.rpm 2.7 MB/s | 803 kB 00:00 2024-05-03T18:34:33.539 INFO:teuthology.orchestra.run.smithi032.stdout:(42/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.0 MB/s | 253 kB 00:00 2024-05-03T18:34:33.590 INFO:teuthology.orchestra.run.smithi032.stdout:(43/152): python3-cryptography-36.0.1-4.el9.x86 12 MB/s | 1.2 MB 00:00 2024-05-03T18:34:33.632 INFO:teuthology.orchestra.run.smithi032.stdout:(44/152): python3-ply-3.11-14.el9.noarch.rpm 1.1 MB/s | 106 kB 00:00 2024-05-03T18:34:33.657 INFO:teuthology.orchestra.run.smithi032.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 2.0 MB/s | 135 kB 00:00 2024-05-03T18:34:33.699 INFO:teuthology.orchestra.run.smithi032.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 1.8 MB/s | 125 kB 00:00 2024-05-03T18:34:33.724 INFO:teuthology.orchestra.run.smithi032.stdout:(47/152): python3-urllib3-1.26.5-5.el9.noarch.r 3.1 MB/s | 215 kB 00:00 2024-05-03T18:34:33.791 INFO:teuthology.orchestra.run.smithi032.stdout:(48/152): automake-1.16.2-8.el9.noarch.rpm 7.3 MB/s | 689 kB 00:00 2024-05-03T18:34:33.833 INFO:teuthology.orchestra.run.smithi032.stdout:(49/152): bison-3.7.4-5.el9.x86_64.rpm 8.5 MB/s | 943 kB 00:00 2024-05-03T18:34:33.875 INFO:teuthology.orchestra.run.smithi032.stdout:(50/152): boost-program-options-1.75.0-8.el9.x8 1.2 MB/s | 107 kB 00:00 2024-05-03T18:34:33.900 INFO:teuthology.orchestra.run.smithi032.stdout:(51/152): elfutils-libelf-devel-0.190-2.el9.x86 320 kB/s | 21 kB 00:00 2024-05-03T18:34:33.951 INFO:teuthology.orchestra.run.smithi032.stdout:(52/152): flex-2.6.4-9.el9.x86_64.rpm 4.1 MB/s | 313 kB 00:00 2024-05-03T18:34:33.976 INFO:teuthology.orchestra.run.smithi032.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 435 kB/s | 33 kB 00:00 2024-05-03T18:34:34.043 INFO:teuthology.orchestra.run.smithi032.stdout:(54/152): flexiblas-openblas-openmp-3.0.4-8.el9 274 kB/s | 18 kB 00:00 2024-05-03T18:34:34.102 INFO:teuthology.orchestra.run.smithi032.stdout:(55/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 20 MB/s | 3.0 MB 00:00 2024-05-03T18:34:34.144 INFO:teuthology.orchestra.run.smithi032.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 111 kB/s | 11 kB 00:00 2024-05-03T18:34:34.161 INFO:teuthology.orchestra.run.smithi032.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 160 kB 00:00 2024-05-03T18:34:34.203 INFO:teuthology.orchestra.run.smithi032.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 772 kB/s | 45 kB 00:00 2024-05-03T18:34:34.236 INFO:teuthology.orchestra.run.smithi032.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 8.6 MB/s | 662 kB 00:00 2024-05-03T18:34:34.297 INFO:teuthology.orchestra.run.smithi113.stdout:(31/152): ceph-mgr-dashboard-19.0.0-3450.g9a047 3.2 MB/s | 3.5 MB 00:01 2024-05-03T18:34:34.320 INFO:teuthology.orchestra.run.smithi032.stdout:(60/152): ceph-mgr-diskprediction-local-19.0.0- 2.1 MB/s | 7.4 MB 00:03 2024-05-03T18:34:34.354 INFO:teuthology.orchestra.run.smithi032.stdout:(61/152): libtool-2.4.6-45.el9.x86_64.rpm 3.8 MB/s | 582 kB 00:00 2024-05-03T18:34:34.379 INFO:teuthology.orchestra.run.smithi032.stdout:(62/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 179 kB/s | 25 kB 00:00 2024-05-03T18:34:34.404 INFO:teuthology.orchestra.run.smithi032.stdout:(63/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 567 kB/s | 47 kB 00:00 2024-05-03T18:34:34.430 INFO:teuthology.orchestra.run.smithi032.stdout:(64/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.8 MB/s | 292 kB 00:00 2024-05-03T18:34:34.447 INFO:teuthology.orchestra.run.smithi113.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3450.g9a 1.8 MB/s | 274 kB 00:00 2024-05-03T18:34:34.454 INFO:teuthology.orchestra.run.smithi032.stdout:(65/152): lua-5.4.4-4.el9.x86_64.rpm 2.4 MB/s | 188 kB 00:00 2024-05-03T18:34:34.480 INFO:teuthology.orchestra.run.smithi032.stdout:(66/152): openblas-0.3.21-2.el9.x86_64.rpm 466 kB/s | 35 kB 00:00 2024-05-03T18:34:34.564 INFO:teuthology.orchestra.run.smithi113.stdout:(33/152): ceph-mgr-rook-19.0.0-3450.g9a047fe7.e 423 kB/s | 49 kB 00:00 2024-05-03T18:34:34.656 INFO:teuthology.orchestra.run.smithi032.stdout:(67/152): openblas-openmp-0.3.21-2.el9.x86_64.r 21 MB/s | 4.7 MB 00:00 2024-05-03T18:34:34.690 INFO:teuthology.orchestra.run.smithi113.stdout:(34/152): ceph-mgr-diskprediction-local-19.0.0- 5.8 MB/s | 7.4 MB 00:01 2024-05-03T18:34:34.715 INFO:teuthology.orchestra.run.smithi113.stdout:(35/152): ceph-prometheus-alerts-19.0.0-3450.g9 109 kB/s | 16 kB 00:00 2024-05-03T18:34:34.739 INFO:teuthology.orchestra.run.smithi032.stdout:(68/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 16 MB/s | 4.4 MB 00:00 2024-05-03T18:34:34.765 INFO:teuthology.orchestra.run.smithi113.stdout:(36/152): ceph-volume-19.0.0-3450.g9a047fe7.el9 3.6 MB/s | 273 kB 00:00 2024-05-03T18:34:34.798 INFO:teuthology.orchestra.run.smithi032.stdout:(69/152): postgresql-contrib-13.14-1.el9.x86_64 6.3 MB/s | 911 kB 00:00 2024-05-03T18:34:34.800 INFO:teuthology.orchestra.run.smithi045.stdout:(10/152): ceph-common-19.0.0-3450.g9a047fe7.el9 2.1 MB/s | 21 MB 00:10 2024-05-03T18:34:34.823 INFO:teuthology.orchestra.run.smithi032.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 1.6 MB/s | 133 kB 00:00 2024-05-03T18:34:34.824 INFO:teuthology.orchestra.run.smithi113.stdout:(37/152): cephfs-top-19.0.0-3450.g9a047fe7.el9. 313 kB/s | 18 kB 00:00 2024-05-03T18:34:34.890 INFO:teuthology.orchestra.run.smithi032.stdout:(71/152): postgresql-13.14-1.el9.x86_64.rpm 3.9 MB/s | 1.6 MB 00:00 2024-05-03T18:34:34.909 INFO:teuthology.orchestra.run.smithi045.stdout:(11/152): cephfs-mirror-19.0.0-3450.g9a047fe7.e 2.0 MB/s | 225 kB 00:00 2024-05-03T18:34:34.932 INFO:teuthology.orchestra.run.smithi032.stdout:(72/152): protobuf-3.14.0-13.el9.x86_64.rpm 9.3 MB/s | 1.0 MB 00:00 2024-05-03T18:34:34.941 INFO:teuthology.orchestra.run.smithi113.stdout:(38/152): cephadm-19.0.0-3450.g9a047fe7.el9.noa 3.3 MB/s | 763 kB 00:00 2024-05-03T18:34:34.951 INFO:teuthology.orchestra.run.smithi045.stdout:(12/152): cephfs-shell-19.0.0-3450.g9a047fe7.el 652 kB/s | 27 kB 00:00 2024-05-03T18:34:34.974 INFO:teuthology.orchestra.run.smithi032.stdout:(73/152): python3-attrs-20.3.0-7.el9.noarch.rpm 1.0 MB/s | 87 kB 00:00 2024-05-03T18:34:34.993 INFO:teuthology.orchestra.run.smithi045.stdout:(13/152): libcephfs-devel-19.0.0-3450.g9a047fe7 762 kB/s | 32 kB 00:00 2024-05-03T18:34:35.041 INFO:teuthology.orchestra.run.smithi113.stdout:(39/152): fuse-2.9.9-15.el9.x86_64.rpm 370 kB/s | 80 kB 00:00 2024-05-03T18:34:35.057 INFO:teuthology.orchestra.run.smithi032.stdout:(74/152): postgresql-server-13.14-1.el9.x86_64. 22 MB/s | 5.8 MB 00:00 2024-05-03T18:34:35.108 INFO:teuthology.orchestra.run.smithi032.stdout:(75/152): python3-devel-3.9.18-3.el9.x86_64.rpm 1.8 MB/s | 244 kB 00:00 2024-05-03T18:34:35.142 INFO:teuthology.orchestra.run.smithi032.stdout:(76/152): python3-jinja2-2.11.3-5.el9.noarch.rp 2.9 MB/s | 248 kB 00:00 2024-05-03T18:34:35.185 INFO:teuthology.orchestra.run.smithi045.stdout:(14/152): libcephfs2-19.0.0-3450.g9a047fe7.el9. 3.8 MB/s | 747 kB 00:00 2024-05-03T18:34:35.219 INFO:teuthology.orchestra.run.smithi113.stdout:(40/152): libquadmath-11.4.1-3.el9.x86_64.rpm 1.2 MB/s | 195 kB 00:00 2024-05-03T18:34:35.225 INFO:teuthology.orchestra.run.smithi032.stdout:(77/152): python3-babel-2.9.1-2.el9.noarch.rpm 20 MB/s | 6.0 MB 00:00 2024-05-03T18:34:35.251 INFO:teuthology.orchestra.run.smithi032.stdout:(78/152): python3-jsonpatch-1.21-16.el9.noarch. 180 kB/s | 26 kB 00:00 2024-05-03T18:34:35.276 INFO:teuthology.orchestra.run.smithi032.stdout:(79/152): python3-jsonpointer-2.0-4.el9.noarch. 143 kB/s | 19 kB 00:00 2024-05-03T18:34:35.277 INFO:teuthology.orchestra.run.smithi045.stdout:(15/152): libcephsqlite-19.0.0-3450.g9a047fe7.e 1.7 MB/s | 163 kB 00:00 2024-05-03T18:34:35.301 INFO:teuthology.orchestra.run.smithi032.stdout:(80/152): python3-mako-1.1.4-6.el9.noarch.rpm 2.2 MB/s | 172 kB 00:00 2024-05-03T18:34:35.327 INFO:teuthology.orchestra.run.smithi032.stdout:(81/152): python3-markupsafe-1.1.1-12.el9.x86_6 460 kB/s | 35 kB 00:00 2024-05-03T18:34:35.352 INFO:teuthology.orchestra.run.smithi045.stdout:(16/152): librados-devel-19.0.0-3450.g9a047fe7. 1.7 MB/s | 127 kB 00:00 2024-05-03T18:34:35.366 INFO:teuthology.orchestra.run.smithi113.stdout:(41/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.5 MB/s | 253 kB 00:00 2024-05-03T18:34:35.385 INFO:teuthology.orchestra.run.smithi032.stdout:(82/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.9 MB/s | 250 kB 00:00 2024-05-03T18:34:35.418 INFO:teuthology.orchestra.run.smithi032.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 2.4 MB/s | 222 kB 00:00 2024-05-03T18:34:35.494 INFO:teuthology.orchestra.run.smithi032.stdout:(84/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 23 MB/s | 5.1 MB 00:00 2024-05-03T18:34:35.519 INFO:teuthology.orchestra.run.smithi032.stdout:(85/152): python3-packaging-20.9-5.el9.noarch.r 575 kB/s | 77 kB 00:00 2024-05-03T18:34:35.527 INFO:teuthology.orchestra.run.smithi045.stdout:(17/152): libradosstriper1-19.0.0-3450.g9a047fe 2.8 MB/s | 500 kB 00:00 2024-05-03T18:34:35.544 INFO:teuthology.orchestra.run.smithi032.stdout:(86/152): python3-prettytable-0.7.2-27.el9.noar 336 kB/s | 42 kB 00:00 2024-05-03T18:34:35.583 INFO:teuthology.orchestra.run.smithi113.stdout:(42/152): libgfortran-11.4.1-3.el9.x86_64.rpm 1.2 MB/s | 803 kB 00:00 2024-05-03T18:34:35.597 INFO:teuthology.orchestra.run.smithi032.stdout:(87/152): python3-protobuf-3.14.0-13.el9.noarch 2.6 MB/s | 269 kB 00:00 2024-05-03T18:34:35.620 INFO:teuthology.orchestra.run.smithi032.stdout:(88/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.5 MB/s | 159 kB 00:00 2024-05-03T18:34:35.645 INFO:teuthology.orchestra.run.smithi032.stdout:(89/152): python3-pyasn1-modules-0.4.8-6.el9.no 2.7 MB/s | 279 kB 00:00 2024-05-03T18:34:35.671 INFO:teuthology.orchestra.run.smithi032.stdout:(90/152): python3-pytz-2021.1-5.el9.noarch.rpm 693 kB/s | 51 kB 00:00 2024-05-03T18:34:35.692 INFO:teuthology.orchestra.run.smithi113.stdout:(43/152): python3-ply-3.11-14.el9.noarch.rpm 983 kB/s | 106 kB 00:00 2024-05-03T18:34:35.696 INFO:teuthology.orchestra.run.smithi032.stdout:(91/152): python3-requests-oauthlib-1.3.0-12.el 710 kB/s | 54 kB 00:00 2024-05-03T18:34:35.746 INFO:teuthology.orchestra.run.smithi032.stdout:(92/152): python3-toml-0.10.2-6.el9.noarch.rpm 554 kB/s | 42 kB 00:00 2024-05-03T18:34:35.796 INFO:teuthology.orchestra.run.smithi032.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 441 kB/s | 44 kB 00:00 2024-05-03T18:34:35.817 INFO:teuthology.orchestra.run.smithi113.stdout:(44/152): python3-cryptography-36.0.1-4.el9.x86 2.7 MB/s | 1.2 MB 00:00 2024-05-03T18:34:35.843 INFO:teuthology.orchestra.run.smithi113.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 899 kB/s | 135 kB 00:00 2024-05-03T18:34:35.847 INFO:teuthology.orchestra.run.smithi032.stdout:(94/152): qatlib-24.02.0-1.el9.x86_64.rpm 2.1 MB/s | 221 kB 00:00 2024-05-03T18:34:35.880 INFO:teuthology.orchestra.run.smithi032.stdout:(95/152): qatlib-service-24.02.0-1.el9.x86_64.r 428 kB/s | 36 kB 00:00 2024-05-03T18:34:35.901 INFO:teuthology.orchestra.run.smithi113.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 1.5 MB/s | 125 kB 00:00 2024-05-03T18:34:35.922 INFO:teuthology.orchestra.run.smithi032.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 629 kB/s | 47 kB 00:00 2024-05-03T18:34:35.990 INFO:teuthology.orchestra.run.smithi032.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 2.7 MB/s | 305 kB 00:00 2024-05-03T18:34:36.015 INFO:teuthology.orchestra.run.smithi032.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 627 kB/s | 58 kB 00:00 2024-05-03T18:34:36.027 INFO:teuthology.orchestra.run.smithi113.stdout:(47/152): python3-urllib3-1.26.5-5.el9.noarch.r 1.1 MB/s | 215 kB 00:00 2024-05-03T18:34:36.102 INFO:teuthology.orchestra.run.smithi113.stdout:(48/152): automake-1.16.2-8.el9.noarch.rpm 3.4 MB/s | 689 kB 00:00 2024-05-03T18:34:36.144 INFO:teuthology.orchestra.run.smithi113.stdout:(49/152): boost-program-options-1.75.0-8.el9.x8 2.5 MB/s | 107 kB 00:00 2024-05-03T18:34:36.186 INFO:teuthology.orchestra.run.smithi113.stdout:(50/152): elfutils-libelf-devel-0.190-2.el9.x86 517 kB/s | 21 kB 00:00 2024-05-03T18:34:36.199 INFO:teuthology.orchestra.run.smithi032.stdout:(99/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 29 MB/s | 16 MB 00:00 2024-05-03T18:34:36.220 INFO:teuthology.orchestra.run.smithi113.stdout:(51/152): bison-3.7.4-5.el9.x86_64.rpm 4.8 MB/s | 943 kB 00:00 2024-05-03T18:34:36.224 INFO:teuthology.orchestra.run.smithi032.stdout:(100/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 400 kB/s | 94 kB 00:00 2024-05-03T18:34:36.245 INFO:teuthology.orchestra.run.smithi113.stdout:(52/152): flex-2.6.4-9.el9.x86_64.rpm 5.2 MB/s | 313 kB 00:00 2024-05-03T18:34:36.249 INFO:teuthology.orchestra.run.smithi032.stdout:(101/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 272 kB/s | 64 kB 00:00 2024-05-03T18:34:36.271 INFO:teuthology.orchestra.run.smithi113.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 652 kB/s | 33 kB 00:00 2024-05-03T18:34:36.274 INFO:teuthology.orchestra.run.smithi032.stdout:(102/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 595 kB/s | 45 kB 00:00 2024-05-03T18:34:36.313 INFO:teuthology.orchestra.run.smithi113.stdout:(54/152): flexiblas-openblas-openmp-3.0.4-8.el9 436 kB/s | 18 kB 00:00 2024-05-03T18:34:36.371 INFO:teuthology.orchestra.run.smithi113.stdout:(55/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 24 MB/s | 3.0 MB 00:00 2024-05-03T18:34:36.397 INFO:teuthology.orchestra.run.smithi113.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 133 kB/s | 11 kB 00:00 2024-05-03T18:34:36.408 INFO:teuthology.orchestra.run.smithi032.stdout:(103/152): abseil-cpp-20211102.0-3.el9.x86_64.r 4.0 MB/s | 548 kB 00:00 2024-05-03T18:34:36.422 INFO:teuthology.orchestra.run.smithi113.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-05-03T18:34:36.447 INFO:teuthology.orchestra.run.smithi113.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 903 kB/s | 45 kB 00:00 2024-05-03T18:34:36.450 INFO:teuthology.orchestra.run.smithi032.stdout:(104/152): gperftools-libs-2.9.1-2.el9.x86_64.r 7.2 MB/s | 309 kB 00:00 2024-05-03T18:34:36.481 INFO:teuthology.orchestra.run.smithi113.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 662 kB 00:00 2024-05-03T18:34:36.500 INFO:teuthology.orchestra.run.smithi032.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 389 kB/s | 19 kB 00:00 2024-05-03T18:34:36.514 INFO:teuthology.orchestra.run.smithi113.stdout:(60/152): libtool-2.4.6-45.el9.x86_64.rpm 8.5 MB/s | 582 kB 00:00 2024-05-03T18:34:36.540 INFO:teuthology.orchestra.run.smithi113.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 434 kB/s | 25 kB 00:00 2024-05-03T18:34:36.565 INFO:teuthology.orchestra.run.smithi113.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 948 kB/s | 47 kB 00:00 2024-05-03T18:34:36.590 INFO:teuthology.orchestra.run.smithi113.stdout:(63/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-05-03T18:34:36.615 INFO:teuthology.orchestra.run.smithi113.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 188 kB 00:00 2024-05-03T18:34:36.641 INFO:teuthology.orchestra.run.smithi113.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 703 kB/s | 35 kB 00:00 2024-05-03T18:34:36.726 INFO:teuthology.orchestra.run.smithi032.stdout:(106/152): libarrow-9.0.0-11.el9.x86_64.rpm 26 MB/s | 4.4 MB 00:00 2024-05-03T18:34:36.741 INFO:teuthology.orchestra.run.smithi113.stdout:(66/152): openblas-openmp-0.3.21-2.el9.x86_64.r 38 MB/s | 4.7 MB 00:00 2024-05-03T18:34:36.768 INFO:teuthology.orchestra.run.smithi032.stdout:(107/152): libarrow-doc-9.0.0-11.el9.noarch.rpm 627 kB/s | 26 kB 00:00 2024-05-03T18:34:36.808 INFO:teuthology.orchestra.run.smithi113.stdout:(67/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 27 MB/s | 4.4 MB 00:00 2024-05-03T18:34:36.810 INFO:teuthology.orchestra.run.smithi032.stdout:(108/152): liboath-2.6.7-2.el9.x86_64.rpm 1.1 MB/s | 49 kB 00:00 2024-05-03T18:34:36.843 INFO:teuthology.orchestra.run.smithi032.stdout:(109/152): libunwind-1.6.2-1.el9.x86_64.rpm 2.0 MB/s | 67 kB 00:00 2024-05-03T18:34:36.850 INFO:teuthology.orchestra.run.smithi113.stdout:(68/152): postgresql-13.14-1.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-05-03T18:34:36.884 INFO:teuthology.orchestra.run.smithi113.stdout:(69/152): postgresql-contrib-13.14-1.el9.x86_64 12 MB/s | 911 kB 00:00 2024-05-03T18:34:36.885 INFO:teuthology.orchestra.run.smithi032.stdout:(110/152): luarocks-3.9.2-1.el9.noarch.rpm 3.5 MB/s | 151 kB 00:00 2024-05-03T18:34:36.909 INFO:teuthology.orchestra.run.smithi113.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 2.2 MB/s | 133 kB 00:00 2024-05-03T18:34:36.910 INFO:teuthology.orchestra.run.smithi032.stdout:(111/152): lua-devel-5.4.4-4.el9.x86_64.rpm 32 kB/s | 22 kB 00:00 2024-05-03T18:34:36.952 INFO:teuthology.orchestra.run.smithi032.stdout:(112/152): parquet-libs-9.0.0-11.el9.x86_64.rpm 12 MB/s | 839 kB 00:00 2024-05-03T18:34:36.959 INFO:teuthology.orchestra.run.smithi113.stdout:(71/152): protobuf-3.14.0-13.el9.x86_64.rpm 20 MB/s | 1.0 MB 00:00 2024-05-03T18:34:36.994 INFO:teuthology.orchestra.run.smithi032.stdout:(113/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.0 MB/s | 43 kB 00:00 2024-05-03T18:34:37.028 INFO:teuthology.orchestra.run.smithi032.stdout:(114/152): python3-asyncssh-2.13.2-5.el9.noarch 4.6 MB/s | 548 kB 00:00 2024-05-03T18:34:37.034 INFO:teuthology.orchestra.run.smithi113.stdout:(72/152): postgresql-server-13.14-1.el9.x86_64. 38 MB/s | 5.8 MB 00:00 2024-05-03T18:34:37.053 INFO:teuthology.orchestra.run.smithi032.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 547 kB/s | 32 kB 00:00 2024-05-03T18:34:37.063 INFO:teuthology.orchestra.run.smithi113.stdout:(73/152): python3-attrs-20.3.0-7.el9.noarch.rpm 873 kB/s | 87 kB 00:00 2024-05-03T18:34:37.079 INFO:teuthology.orchestra.run.smithi032.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 298 kB/s | 15 kB 00:00 2024-05-03T18:34:37.102 INFO:teuthology.orchestra.run.smithi113.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 5.7 MB/s | 244 kB 00:00 2024-05-03T18:34:37.104 INFO:teuthology.orchestra.run.smithi032.stdout:(117/152): python3-cheroot-8.6.0-4.el9.noarch.r 3.3 MB/s | 172 kB 00:00 2024-05-03T18:34:37.130 INFO:teuthology.orchestra.run.smithi032.stdout:(118/152): python3-cherrypy-18.6.1-2.el9.noarch 6.9 MB/s | 358 kB 00:00 2024-05-03T18:34:37.163 INFO:teuthology.orchestra.run.smithi032.stdout:(119/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.0 MB/s | 303 kB 00:00 2024-05-03T18:34:37.180 INFO:teuthology.orchestra.run.smithi032.stdout:(120/152): python3-colorama-0.4.4-12.el9.noarch 702 kB/s | 35 kB 00:00 2024-05-03T18:34:37.185 INFO:teuthology.orchestra.run.smithi113.stdout:(75/152): python3-babel-2.9.1-2.el9.noarch.rpm 40 MB/s | 6.0 MB 00:00 2024-05-03T18:34:37.211 INFO:teuthology.orchestra.run.smithi113.stdout:(76/152): python3-jinja2-2.11.3-5.el9.noarch.rp 2.2 MB/s | 248 kB 00:00 2024-05-03T18:34:37.214 INFO:teuthology.orchestra.run.smithi032.stdout:(121/152): python3-google-auth-2.29.0-1.el9.noa 4.0 MB/s | 206 kB 00:00 2024-05-03T18:34:37.228 INFO:teuthology.orchestra.run.smithi113.stdout:(77/152): python3-jsonpatch-1.21-16.el9.noarch. 614 kB/s | 26 kB 00:00 2024-05-03T18:34:37.245 INFO:teuthology.orchestra.run.smithi113.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 571 kB/s | 19 kB 00:00 2024-05-03T18:34:37.270 INFO:teuthology.orchestra.run.smithi113.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 4.0 MB/s | 172 kB 00:00 2024-05-03T18:34:37.281 INFO:teuthology.orchestra.run.smithi032.stdout:(122/152): python3-grpcio-1.46.7-10.el9.x86_64. 20 MB/s | 2.0 MB 00:00 2024-05-03T18:34:37.295 INFO:teuthology.orchestra.run.smithi113.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 692 kB/s | 35 kB 00:00 2024-05-03T18:34:37.306 INFO:teuthology.orchestra.run.smithi032.stdout:(123/152): python3-grpcio-tools-1.46.7-10.el9.x 1.5 MB/s | 144 kB 00:00 2024-05-03T18:34:37.337 INFO:teuthology.orchestra.run.smithi113.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 5.9 MB/s | 250 kB 00:00 2024-05-03T18:34:37.348 INFO:teuthology.orchestra.run.smithi032.stdout:(124/152): protobuf-compiler-3.14.0-13.el9.x86_ 786 kB/s | 863 kB 00:01 2024-05-03T18:34:37.373 INFO:teuthology.orchestra.run.smithi032.stdout:(125/152): python3-jaraco-8.2.1-3.el9.noarch.rp 116 kB/s | 11 kB 00:00 2024-05-03T18:34:37.399 INFO:teuthology.orchestra.run.smithi032.stdout:(126/152): python3-jaraco-classes-3.2.1-5.el9.n 191 kB/s | 18 kB 00:00 2024-05-03T18:34:37.404 INFO:teuthology.orchestra.run.smithi113.stdout:(82/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 38 MB/s | 5.1 MB 00:00 2024-05-03T18:34:37.424 INFO:teuthology.orchestra.run.smithi032.stdout:(127/152): python3-jaraco-collections-3.0.0-8.e 307 kB/s | 23 kB 00:00 2024-05-03T18:34:37.430 INFO:teuthology.orchestra.run.smithi113.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 2.4 MB/s | 222 kB 00:00 2024-05-03T18:34:37.449 INFO:teuthology.orchestra.run.smithi032.stdout:(128/152): python3-jaraco-functools-3.5.0-2.el9 257 kB/s | 19 kB 00:00 2024-05-03T18:34:37.455 INFO:teuthology.orchestra.run.smithi113.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-05-03T18:34:37.475 INFO:teuthology.orchestra.run.smithi032.stdout:(129/152): python3-jaraco-text-3.2.0-6.el9.noar 260 kB/s | 20 kB 00:00 2024-05-03T18:34:37.480 INFO:teuthology.orchestra.run.smithi113.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 840 kB/s | 42 kB 00:00 2024-05-03T18:34:37.500 INFO:teuthology.orchestra.run.smithi032.stdout:(130/152): python3-logutils-0.3.5-21.el9.noarch 920 kB/s | 46 kB 00:00 2024-05-03T18:34:37.505 INFO:teuthology.orchestra.run.smithi113.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 5.3 MB/s | 269 kB 00:00 2024-05-03T18:34:37.531 INFO:teuthology.orchestra.run.smithi113.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-05-03T18:34:37.534 INFO:teuthology.orchestra.run.smithi032.stdout:(131/152): python3-kubernetes-26.1.0-2.el9.noar 9.3 MB/s | 1.0 MB 00:00 2024-05-03T18:34:37.556 INFO:teuthology.orchestra.run.smithi113.stdout:(88/152): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-05-03T18:34:37.559 INFO:teuthology.orchestra.run.smithi032.stdout:(132/152): python3-natsort-7.1.1-5.el9.noarch.r 985 kB/s | 58 kB 00:00 2024-05-03T18:34:37.576 INFO:teuthology.orchestra.run.smithi032.stdout:(133/152): python3-more-itertools-8.12.0-2.el9. 783 kB/s | 79 kB 00:00 2024-05-03T18:34:37.581 INFO:teuthology.orchestra.run.smithi113.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-05-03T18:34:37.601 INFO:teuthology.orchestra.run.smithi032.stdout:(134/152): python3-pecan-1.4.2-3.el9.noarch.rpm 4.0 MB/s | 272 kB 00:00 2024-05-03T18:34:37.606 INFO:teuthology.orchestra.run.smithi113.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-05-03T18:34:37.627 INFO:teuthology.orchestra.run.smithi032.stdout:(135/152): python3-portend-3.1.0-2.el9.noarch.r 244 kB/s | 16 kB 00:00 2024-05-03T18:34:37.649 INFO:teuthology.orchestra.run.smithi113.stdout:(91/152): python3-toml-0.10.2-6.el9.noarch.rpm 997 kB/s | 42 kB 00:00 2024-05-03T18:34:37.652 INFO:teuthology.orchestra.run.smithi032.stdout:(136/152): python3-pyperclip-1.8.2-2.el9.noarch 563 kB/s | 28 kB 00:00 2024-05-03T18:34:37.678 INFO:teuthology.orchestra.run.smithi032.stdout:(137/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 891 kB/s | 90 kB 00:00 2024-05-03T18:34:37.691 INFO:teuthology.orchestra.run.smithi113.stdout:(92/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 1.0 MB/s | 44 kB 00:00 2024-05-03T18:34:37.694 INFO:teuthology.orchestra.run.smithi045.stdout:(18/152): ceph-test-19.0.0-3450.g9a047fe7.el9.x 6.6 MB/s | 49 MB 00:07 2024-05-03T18:34:37.702 INFO:teuthology.orchestra.run.smithi032.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 409 kB/s | 31 kB 00:00 2024-05-03T18:34:37.728 INFO:teuthology.orchestra.run.smithi032.stdout:(139/152): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-05-03T18:34:37.732 INFO:teuthology.orchestra.run.smithi113.stdout:(93/152): qatlib-24.02.0-1.el9.x86_64.rpm 5.2 MB/s | 221 kB 00:00 2024-05-03T18:34:37.753 INFO:teuthology.orchestra.run.smithi032.stdout:(140/152): python3-rsa-4.9-2.el9.noarch.rpm 1.2 MB/s | 59 kB 00:00 2024-05-03T18:34:37.769 INFO:teuthology.orchestra.run.smithi045.stdout:(19/152): librgw2-19.0.0-3450.g9a047fe7.el9.x86 2.4 MB/s | 5.4 MB 00:02 2024-05-03T18:34:37.775 INFO:teuthology.orchestra.run.smithi113.stdout:(94/152): qatlib-service-24.02.0-1.el9.x86_64.r 864 kB/s | 36 kB 00:00 2024-05-03T18:34:37.778 INFO:teuthology.orchestra.run.smithi032.stdout:(141/152): python3-tempora-5.0.0-2.el9.noarch.r 474 kB/s | 36 kB 00:00 2024-05-03T18:34:37.794 INFO:teuthology.orchestra.run.smithi045.stdout:(20/152): python3-ceph-argparse-19.0.0-3450.g9a 452 kB/s | 45 kB 00:00 2024-05-03T18:34:37.803 INFO:teuthology.orchestra.run.smithi032.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 675 kB/s | 51 kB 00:00 2024-05-03T18:34:37.829 INFO:teuthology.orchestra.run.smithi032.stdout:(143/152): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-05-03T18:34:37.854 INFO:teuthology.orchestra.run.smithi032.stdout:(144/152): python3-webob-1.8.7-6.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-05-03T18:34:37.879 INFO:teuthology.orchestra.run.smithi032.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 399 kB/s | 20 kB 00:00 2024-05-03T18:34:37.905 INFO:teuthology.orchestra.run.smithi032.stdout:(146/152): python3-werkzeug-2.0.3-3.el9.1.noarc 4.1 MB/s | 427 kB 00:00 2024-05-03T18:34:37.930 INFO:teuthology.orchestra.run.smithi032.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 2.5 MB/s | 192 kB 00:00 2024-05-03T18:34:37.933 INFO:teuthology.orchestra.run.smithi113.stdout:(95/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 46 MB/s | 16 MB 00:00 2024-05-03T18:34:37.959 INFO:teuthology.orchestra.run.smithi113.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 258 kB/s | 47 kB 00:00 2024-05-03T18:34:37.961 INFO:teuthology.orchestra.run.smithi045.stdout:(21/152): ceph-osd-19.0.0-3450.g9a047fe7.el9.x8 1.7 MB/s | 17 MB 00:09 2024-05-03T18:34:37.972 INFO:teuthology.orchestra.run.smithi032.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 17 MB/s | 1.6 MB 00:00 2024-05-03T18:34:37.984 INFO:teuthology.orchestra.run.smithi113.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 6.0 MB/s | 305 kB 00:00 2024-05-03T18:34:37.987 INFO:teuthology.orchestra.run.smithi045.stdout:(22/152): python3-ceph-common-19.0.0-3450.g9a04 624 kB/s | 135 kB 00:00 2024-05-03T18:34:38.009 INFO:teuthology.orchestra.run.smithi113.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-05-03T18:34:38.029 INFO:teuthology.orchestra.run.smithi045.stdout:(23/152): python3-cephfs-19.0.0-3450.g9a047fe7. 690 kB/s | 161 kB 00:00 2024-05-03T18:34:38.034 INFO:teuthology.orchestra.run.smithi113.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.8 MB/s | 94 kB 00:00 2024-05-03T18:34:38.060 INFO:teuthology.orchestra.run.smithi113.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-05-03T18:34:38.085 INFO:teuthology.orchestra.run.smithi113.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 889 kB/s | 45 kB 00:00 2024-05-03T18:34:38.137 INFO:teuthology.orchestra.run.smithi045.stdout:(24/152): python3-rgw-19.0.0-3450.g9a047fe7.el9 919 kB/s | 100 kB 00:00 2024-05-03T18:34:38.162 INFO:teuthology.orchestra.run.smithi045.stdout:(25/152): python3-rados-19.0.0-3450.g9a047fe7.e 1.6 MB/s | 320 kB 00:00 2024-05-03T18:34:38.177 INFO:teuthology.orchestra.run.smithi113.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 190 kB/s | 22 kB 00:00 2024-05-03T18:34:38.188 INFO:teuthology.orchestra.run.smithi045.stdout:(26/152): python3-rbd-19.0.0-3450.g9a047fe7.el9 1.5 MB/s | 298 kB 00:00 2024-05-03T18:34:38.189 INFO:teuthology.orchestra.run.smithi032.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 6.2 MB/s | 1.3 MB 00:00 2024-05-03T18:34:38.255 INFO:teuthology.orchestra.run.smithi045.stdout:(27/152): rbd-fuse-19.0.0-3450.g9a047fe7.el9.x8 729 kB/s | 85 kB 00:00 2024-05-03T18:34:38.291 INFO:teuthology.orchestra.run.smithi045.stdout:(28/152): rbd-nbd-19.0.0-3450.g9a047fe7.el9.x86 1.7 MB/s | 170 kB 00:00 2024-05-03T18:34:38.302 INFO:teuthology.orchestra.run.smithi113.stdout:(103/152): protobuf-compiler-3.14.0-13.el9.x86_ 3.9 MB/s | 863 kB 00:00 2024-05-03T18:34:38.306 INFO:teuthology.orchestra.run.smithi032.stdout:(150/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 21 MB/s | 2.4 MB 00:00 2024-05-03T18:34:38.364 INFO:teuthology.orchestra.run.smithi045.stdout:(29/152): ceph-grafana-dashboards-19.0.0-3450.g 315 kB/s | 34 kB 00:00 2024-05-03T18:34:38.389 INFO:teuthology.orchestra.run.smithi045.stdout:(30/152): ceph-mgr-cephadm-19.0.0-3450.g9a047fe 1.4 MB/s | 142 kB 00:00 2024-05-03T18:34:38.528 INFO:teuthology.orchestra.run.smithi113.stdout:(104/152): abseil-cpp-20211102.0-3.el9.x86_64.r 1.5 MB/s | 548 kB 00:00 2024-05-03T18:34:38.586 INFO:teuthology.orchestra.run.smithi113.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 334 kB/s | 19 kB 00:00 2024-05-03T18:34:38.612 INFO:teuthology.orchestra.run.smithi113.stdout:(106/152): gperftools-libs-2.9.1-2.el9.x86_64.r 1.0 MB/s | 309 kB 00:00 2024-05-03T18:34:38.670 INFO:teuthology.orchestra.run.smithi113.stdout:(107/152): libarrow-doc-9.0.0-11.el9.noarch.rpm 446 kB/s | 26 kB 00:00 2024-05-03T18:34:38.729 INFO:teuthology.orchestra.run.smithi113.stdout:(108/152): liboath-2.6.7-2.el9.x86_64.rpm 832 kB/s | 49 kB 00:00 2024-05-03T18:34:38.788 INFO:teuthology.orchestra.run.smithi113.stdout:(109/152): libunwind-1.6.2-1.el9.x86_64.rpm 1.1 MB/s | 67 kB 00:00 2024-05-03T18:34:38.846 INFO:teuthology.orchestra.run.smithi113.stdout:(110/152): luarocks-3.9.2-1.el9.noarch.rpm 2.5 MB/s | 151 kB 00:00 2024-05-03T18:34:38.980 INFO:teuthology.orchestra.run.smithi113.stdout:(111/152): parquet-libs-9.0.0-11.el9.x86_64.rpm 6.1 MB/s | 839 kB 00:00 2024-05-03T18:34:39.038 INFO:teuthology.orchestra.run.smithi113.stdout:(112/152): libarrow-9.0.0-11.el9.x86_64.rpm 9.8 MB/s | 4.4 MB 00:00 2024-05-03T18:34:39.072 INFO:teuthology.orchestra.run.smithi113.stdout:(113/152): python3-asyncssh-2.13.2-5.el9.noarch 5.8 MB/s | 548 kB 00:00 2024-05-03T18:34:39.097 INFO:teuthology.orchestra.run.smithi113.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 740 kB/s | 43 kB 00:00 2024-05-03T18:34:39.131 INFO:teuthology.orchestra.run.smithi113.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 548 kB/s | 32 kB 00:00 2024-05-03T18:34:39.156 INFO:teuthology.orchestra.run.smithi113.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 255 kB/s | 15 kB 00:00 2024-05-03T18:34:39.190 INFO:teuthology.orchestra.run.smithi113.stdout:(117/152): python3-cheroot-8.6.0-4.el9.noarch.r 2.9 MB/s | 172 kB 00:00 2024-05-03T18:34:39.222 INFO:teuthology.orchestra.run.smithi045.stdout:(31/152): ceph-mgr-dashboard-19.0.0-3450.g9a047 4.1 MB/s | 3.5 MB 00:00 2024-05-03T18:34:39.223 INFO:teuthology.orchestra.run.smithi113.stdout:(118/152): python3-cherrypy-18.6.1-2.el9.noarch 5.2 MB/s | 358 kB 00:00 2024-05-03T18:34:39.256 INFO:teuthology.orchestra.run.smithi032.stdout:(151/152): librbd1-19.0.0-3450.g9a047fe7.el9.x8 2.4 MB/s | 3.1 MB 00:01 2024-05-03T18:34:39.265 INFO:teuthology.orchestra.run.smithi113.stdout:(119/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.9 MB/s | 303 kB 00:00 2024-05-03T18:34:39.290 INFO:teuthology.orchestra.run.smithi113.stdout:(120/152): python3-colorama-0.4.4-12.el9.noarch 527 kB/s | 35 kB 00:00 2024-05-03T18:34:39.314 INFO:teuthology.orchestra.run.smithi045.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3450.g9a 2.9 MB/s | 274 kB 00:00 2024-05-03T18:34:39.324 INFO:teuthology.orchestra.run.smithi113.stdout:(121/152): python3-google-auth-2.29.0-1.el9.noa 3.4 MB/s | 206 kB 00:00 2024-05-03T18:34:39.373 INFO:teuthology.orchestra.run.smithi045.stdout:(33/152): rbd-mirror-19.0.0-3450.g9a047fe7.el9. 2.6 MB/s | 3.1 MB 00:01 2024-05-03T18:34:39.374 INFO:teuthology.orchestra.run.smithi113.stdout:(122/152): python3-grpcio-tools-1.46.7-10.el9.x 2.8 MB/s | 144 kB 00:00 2024-05-03T18:34:39.398 INFO:teuthology.orchestra.run.smithi045.stdout:(34/152): ceph-mgr-rook-19.0.0-3450.g9a047fe7.e 591 kB/s | 49 kB 00:00 2024-05-03T18:34:39.425 INFO:teuthology.orchestra.run.smithi113.stdout:(123/152): python3-jaraco-8.2.1-3.el9.noarch.rp 214 kB/s | 11 kB 00:00 2024-05-03T18:34:39.448 INFO:teuthology.orchestra.run.smithi045.stdout:(35/152): ceph-prometheus-alerts-19.0.0-3450.g9 219 kB/s | 16 kB 00:00 2024-05-03T18:34:39.484 INFO:teuthology.orchestra.run.smithi113.stdout:(124/152): python3-jaraco-classes-3.2.1-5.el9.n 303 kB/s | 18 kB 00:00 2024-05-03T18:34:39.490 INFO:teuthology.orchestra.run.smithi045.stdout:(36/152): ceph-volume-19.0.0-3450.g9a047fe7.el9 2.9 MB/s | 273 kB 00:00 2024-05-03T18:34:39.559 INFO:teuthology.orchestra.run.smithi113.stdout:(125/152): python3-grpcio-1.46.7-10.el9.x86_64. 7.6 MB/s | 2.0 MB 00:00 2024-05-03T18:34:39.574 INFO:teuthology.orchestra.run.smithi045.stdout:(37/152): cephfs-top-19.0.0-3450.g9a047fe7.el9. 219 kB/s | 18 kB 00:00 2024-05-03T18:34:39.576 INFO:teuthology.orchestra.run.smithi113.stdout:(126/152): python3-jaraco-collections-3.0.0-8.e 251 kB/s | 23 kB 00:00 2024-05-03T18:34:39.609 INFO:teuthology.orchestra.run.smithi113.stdout:(127/152): python3-jaraco-functools-3.5.0-2.el9 386 kB/s | 19 kB 00:00 2024-05-03T18:34:39.626 INFO:teuthology.orchestra.run.smithi113.stdout:(128/152): python3-jaraco-text-3.2.0-6.el9.noar 390 kB/s | 20 kB 00:00 2024-05-03T18:34:39.683 INFO:teuthology.orchestra.run.smithi045.stdout:(38/152): cephadm-19.0.0-3450.g9a047fe7.el9.noa 3.2 MB/s | 763 kB 00:00 2024-05-03T18:34:39.685 INFO:teuthology.orchestra.run.smithi113.stdout:(129/152): python3-logutils-0.3.5-21.el9.noarch 793 kB/s | 46 kB 00:00 2024-05-03T18:34:39.727 INFO:teuthology.orchestra.run.smithi113.stdout:(130/152): python3-kubernetes-26.1.0-2.el9.noar 8.7 MB/s | 1.0 MB 00:00 2024-05-03T18:34:39.752 INFO:teuthology.orchestra.run.smithi113.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 1.1 MB/s | 79 kB 00:00 2024-05-03T18:34:39.783 INFO:teuthology.orchestra.run.smithi045.stdout:(39/152): fuse-2.9.9-15.el9.x86_64.rpm 385 kB/s | 80 kB 00:00 2024-05-03T18:34:39.786 INFO:teuthology.orchestra.run.smithi113.stdout:(132/152): python3-natsort-7.1.1-5.el9.noarch.r 986 kB/s | 58 kB 00:00 2024-05-03T18:34:39.790 INFO:teuthology.orchestra.run.smithi032.stdout:(152/152): librados2-19.0.0-3450.g9a047fe7.el9. 1.8 MB/s | 3.4 MB 00:01 2024-05-03T18:34:39.794 INFO:teuthology.orchestra.run.smithi032.stdout:-------------------------------------------------------------------------------- 2024-05-03T18:34:39.795 INFO:teuthology.orchestra.run.smithi032.stdout:Total 12 MB/s | 222 MB 00:17 2024-05-03T18:34:39.812 INFO:teuthology.orchestra.run.smithi113.stdout:(133/152): python3-pecan-1.4.2-3.el9.noarch.rpm 4.5 MB/s | 272 kB 00:00 2024-05-03T18:34:39.837 INFO:teuthology.orchestra.run.smithi113.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 327 kB/s | 16 kB 00:00 2024-05-03T18:34:39.867 INFO:teuthology.orchestra.run.smithi045.stdout:(40/152): libquadmath-11.4.1-3.el9.x86_64.rpm 2.3 MB/s | 195 kB 00:00 2024-05-03T18:34:39.870 INFO:teuthology.orchestra.run.smithi113.stdout:(135/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.5 MB/s | 90 kB 00:00 2024-05-03T18:34:39.895 INFO:teuthology.orchestra.run.smithi113.stdout:(136/152): python3-pyperclip-1.8.2-2.el9.noarch 481 kB/s | 28 kB 00:00 2024-05-03T18:34:39.929 INFO:teuthology.orchestra.run.smithi113.stdout:(137/152): python3-repoze-lru-0.7-16.el9.noarch 526 kB/s | 31 kB 00:00 2024-05-03T18:34:39.933 INFO:teuthology.orchestra.run.smithi045.stdout:(41/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.7 MB/s | 253 kB 00:00 2024-05-03T18:34:39.963 INFO:teuthology.orchestra.run.smithi113.stdout:(138/152): python3-routes-2.5.1-5.el9.noarch.rp 2.7 MB/s | 188 kB 00:00 2024-05-03T18:34:39.975 INFO:teuthology.orchestra.run.smithi045.stdout:(42/152): libgfortran-11.4.1-3.el9.x86_64.rpm 2.7 MB/s | 803 kB 00:00 2024-05-03T18:34:39.988 INFO:teuthology.orchestra.run.smithi113.stdout:(139/152): python3-rsa-4.9-2.el9.noarch.rpm 1.0 MB/s | 59 kB 00:00 2024-05-03T18:34:40.022 INFO:teuthology.orchestra.run.smithi113.stdout:(140/152): python3-tempora-5.0.0-2.el9.noarch.r 613 kB/s | 36 kB 00:00 2024-05-03T18:34:40.034 INFO:teuthology.orchestra.run.smithi045.stdout:(43/152): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-05-03T18:34:40.048 INFO:teuthology.orchestra.run.smithi113.stdout:(141/152): python3-typing-extensions-4.4.0-2.el 869 kB/s | 51 kB 00:00 2024-05-03T18:34:40.068 INFO:teuthology.orchestra.run.smithi045.stdout:(44/152): python3-cryptography-36.0.1-4.el9.x86 9.2 MB/s | 1.2 MB 00:00 2024-05-03T18:34:40.080 INFO:teuthology.orchestra.run.smithi113.stdout:(142/152): python3-webob-1.8.7-6.el9.noarch.rpm 3.8 MB/s | 230 kB 00:00 2024-05-03T18:34:40.101 INFO:teuthology.orchestra.run.smithi045.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 2.0 MB/s | 135 kB 00:00 2024-05-03T18:34:40.106 INFO:teuthology.orchestra.run.smithi113.stdout:(143/152): python3-websocket-client-1.2.3-2.el9 1.5 MB/s | 90 kB 00:00 2024-05-03T18:34:40.126 INFO:teuthology.orchestra.run.smithi045.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-05-03T18:34:40.148 INFO:teuthology.orchestra.run.smithi113.stdout:(144/152): python3-werkzeug-2.0.3-3.el9.1.noarc 6.2 MB/s | 427 kB 00:00 2024-05-03T18:34:40.168 INFO:teuthology.orchestra.run.smithi045.stdout:(47/152): python3-urllib3-1.26.5-5.el9.noarch.r 3.1 MB/s | 215 kB 00:00 2024-05-03T18:34:40.173 INFO:teuthology.orchestra.run.smithi113.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 299 kB/s | 20 kB 00:00 2024-05-03T18:34:40.207 INFO:teuthology.orchestra.run.smithi113.stdout:(146/152): re2-20211101-3.el9.x86_64.rpm 3.2 MB/s | 192 kB 00:00 2024-05-03T18:34:40.285 INFO:teuthology.orchestra.run.smithi045.stdout:(48/152): automake-1.16.2-8.el9.noarch.rpm 4.2 MB/s | 689 kB 00:00 2024-05-03T18:34:40.315 INFO:teuthology.orchestra.run.smithi113.stdout:(147/152): thrift-0.15.0-2.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-05-03T18:34:40.327 INFO:teuthology.orchestra.run.smithi045.stdout:(49/152): bison-3.7.4-5.el9.x86_64.rpm 5.8 MB/s | 943 kB 00:00 2024-05-03T18:34:40.377 INFO:teuthology.orchestra.run.smithi045.stdout:(50/152): elfutils-libelf-devel-0.190-2.el9.x86 431 kB/s | 21 kB 00:00 2024-05-03T18:34:40.403 INFO:teuthology.orchestra.run.smithi045.stdout:(51/152): boost-program-options-1.75.0-8.el9.x8 913 kB/s | 107 kB 00:00 2024-05-03T18:34:40.453 INFO:teuthology.orchestra.run.smithi045.stdout:(52/152): flexiblas-3.0.4-8.el9.x86_64.rpm 657 kB/s | 33 kB 00:00 2024-05-03T18:34:40.487 INFO:teuthology.orchestra.run.smithi045.stdout:(53/152): flex-2.6.4-9.el9.x86_64.rpm 2.8 MB/s | 313 kB 00:00 2024-05-03T18:34:40.538 INFO:teuthology.orchestra.run.smithi045.stdout:(54/152): flexiblas-openblas-openmp-3.0.4-8.el9 363 kB/s | 18 kB 00:00 2024-05-03T18:34:40.579 INFO:teuthology.orchestra.run.smithi045.stdout:(55/152): libaio-devel-0.3.111-13.el9.x86_64.rp 268 kB/s | 11 kB 00:00 2024-05-03T18:34:40.646 INFO:teuthology.orchestra.run.smithi045.stdout:(56/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 16 MB/s | 3.0 MB 00:00 2024-05-03T18:34:40.688 INFO:teuthology.orchestra.run.smithi045.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.4 MB/s | 160 kB 00:00 2024-05-03T18:34:40.713 INFO:teuthology.orchestra.run.smithi045.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 677 kB/s | 45 kB 00:00 2024-05-03T18:34:40.797 INFO:teuthology.orchestra.run.smithi045.stdout:(59/152): ceph-mgr-diskprediction-local-19.0.0- 3.1 MB/s | 7.4 MB 00:02 2024-05-03T18:34:40.872 INFO:teuthology.orchestra.run.smithi045.stdout:(60/152): librdkafka-1.6.1-102.el9.x86_64.rpm 3.5 MB/s | 662 kB 00:00 2024-05-03T18:34:40.890 INFO:teuthology.orchestra.run.smithi032.stdout:Running transaction check 2024-05-03T18:34:40.897 INFO:teuthology.orchestra.run.smithi045.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 254 kB/s | 25 kB 00:00 2024-05-03T18:34:40.956 INFO:teuthology.orchestra.run.smithi045.stdout:(62/152): libtool-2.4.6-45.el9.x86_64.rpm 2.3 MB/s | 582 kB 00:00 2024-05-03T18:34:40.981 INFO:teuthology.orchestra.run.smithi045.stdout:(63/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 436 kB/s | 47 kB 00:00 2024-05-03T18:34:41.015 INFO:teuthology.orchestra.run.smithi045.stdout:(64/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.4 MB/s | 292 kB 00:00 2024-05-03T18:34:41.049 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction check succeeded. 2024-05-03T18:34:41.049 INFO:teuthology.orchestra.run.smithi032.stdout:Running transaction test 2024-05-03T18:34:41.082 INFO:teuthology.orchestra.run.smithi045.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 351 kB/s | 35 kB 00:00 2024-05-03T18:34:41.107 INFO:teuthology.orchestra.run.smithi045.stdout:(66/152): lua-5.4.4-4.el9.x86_64.rpm 1.2 MB/s | 188 kB 00:00 2024-05-03T18:34:41.150 INFO:teuthology.orchestra.run.smithi113.stdout:(148/152): librbd1-19.0.0-3450.g9a047fe7.el9.x8 3.8 MB/s | 3.1 MB 00:00 2024-05-03T18:34:41.274 INFO:teuthology.orchestra.run.smithi045.stdout:(67/152): postgresql-13.14-1.el9.x86_64.rpm 9.5 MB/s | 1.6 MB 00:00 2024-05-03T18:34:41.299 INFO:teuthology.orchestra.run.smithi113.stdout:(149/152): librados2-19.0.0-3450.g9a047fe7.el9. 3.1 MB/s | 3.4 MB 00:01 2024-05-03T18:34:41.332 INFO:teuthology.orchestra.run.smithi045.stdout:(68/152): openblas-openmp-0.3.21-2.el9.x86_64.r 15 MB/s | 4.7 MB 00:00 2024-05-03T18:34:41.399 INFO:teuthology.orchestra.run.smithi045.stdout:(69/152): postgresql-private-libs-13.14-1.el9.x 2.0 MB/s | 133 kB 00:00 2024-05-03T18:34:41.433 INFO:teuthology.orchestra.run.smithi045.stdout:(70/152): postgresql-contrib-13.14-1.el9.x86_64 5.6 MB/s | 911 kB 00:00 2024-05-03T18:34:41.492 INFO:teuthology.orchestra.run.smithi045.stdout:(71/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 11 MB/s | 4.4 MB 00:00 2024-05-03T18:34:41.550 INFO:teuthology.orchestra.run.smithi045.stdout:(72/152): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-05-03T18:34:41.617 INFO:teuthology.orchestra.run.smithi045.stdout:(73/152): protobuf-3.14.0-13.el9.x86_64.rpm 5.5 MB/s | 1.0 MB 00:00 2024-05-03T18:34:41.691 INFO:teuthology.orchestra.run.smithi113.stdout:(150/152): openssl-3.2.1-1.el9.x86_64.rpm 2.5 MB/s | 1.3 MB 00:00 2024-05-03T18:34:41.692 INFO:teuthology.orchestra.run.smithi045.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 3.2 MB/s | 244 kB 00:00 2024-05-03T18:34:41.793 INFO:teuthology.orchestra.run.smithi045.stdout:(75/152): postgresql-server-13.14-1.el9.x86_64. 15 MB/s | 5.8 MB 00:00 2024-05-03T18:34:41.843 INFO:teuthology.orchestra.run.smithi045.stdout:(76/152): python3-jinja2-2.11.3-5.el9.noarch.rp 1.6 MB/s | 248 kB 00:00 2024-05-03T18:34:41.868 INFO:teuthology.orchestra.run.smithi045.stdout:(77/152): python3-jsonpatch-1.21-16.el9.noarch. 341 kB/s | 26 kB 00:00 2024-05-03T18:34:41.910 INFO:teuthology.orchestra.run.smithi045.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 287 kB/s | 19 kB 00:00 2024-05-03T18:34:41.952 INFO:teuthology.orchestra.run.smithi045.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 2.0 MB/s | 172 kB 00:00 2024-05-03T18:34:42.103 INFO:teuthology.orchestra.run.smithi045.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 181 kB/s | 35 kB 00:00 2024-05-03T18:34:42.187 INFO:teuthology.orchestra.run.smithi045.stdout:(81/152): python3-babel-2.9.1-2.el9.noarch.rpm 9.4 MB/s | 6.0 MB 00:00 2024-05-03T18:34:42.192 INFO:teuthology.orchestra.run.smithi113.stdout:(151/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 2.7 MB/s | 2.4 MB 00:00 2024-05-03T18:34:42.253 INFO:teuthology.orchestra.run.smithi045.stdout:(82/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.6 MB/s | 250 kB 00:00 2024-05-03T18:34:42.470 INFO:teuthology.orchestra.run.smithi045.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 784 kB/s | 222 kB 00:00 2024-05-03T18:34:42.571 INFO:teuthology.orchestra.run.smithi045.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 243 kB/s | 77 kB 00:00 2024-05-03T18:34:42.596 INFO:teuthology.orchestra.run.smithi045.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 336 kB/s | 42 kB 00:00 2024-05-03T18:34:42.663 INFO:teuthology.orchestra.run.smithi045.stdout:(86/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 7.2 MB/s | 5.1 MB 00:00 2024-05-03T18:34:42.713 INFO:teuthology.orchestra.run.smithi045.stdout:(87/152): python3-protobuf-3.14.0-13.el9.noarch 1.9 MB/s | 269 kB 00:00 2024-05-03T18:34:42.746 INFO:teuthology.orchestra.run.smithi045.stdout:(88/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.0 MB/s | 159 kB 00:00 2024-05-03T18:34:42.772 INFO:teuthology.orchestra.run.smithi045.stdout:(89/152): python3-pyasn1-modules-0.4.8-6.el9.no 2.5 MB/s | 279 kB 00:00 2024-05-03T18:34:42.797 INFO:teuthology.orchestra.run.smithi045.stdout:(90/152): python3-pytz-2021.1-5.el9.noarch.rpm 609 kB/s | 51 kB 00:00 2024-05-03T18:34:42.822 INFO:teuthology.orchestra.run.smithi045.stdout:(91/152): python3-requests-oauthlib-1.3.0-12.el 711 kB/s | 54 kB 00:00 2024-05-03T18:34:42.872 INFO:teuthology.orchestra.run.smithi045.stdout:(92/152): python3-toml-0.10.2-6.el9.noarch.rpm 553 kB/s | 42 kB 00:00 2024-05-03T18:34:42.901 INFO:teuthology.orchestra.run.smithi045.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 590 kB/s | 44 kB 00:00 2024-05-03T18:34:42.927 INFO:teuthology.orchestra.run.smithi032.stdout:Transaction test succeeded. 2024-05-03T18:34:42.928 INFO:teuthology.orchestra.run.smithi032.stdout:Running transaction 2024-05-03T18:34:42.948 INFO:teuthology.orchestra.run.smithi045.stdout:(94/152): qatlib-service-24.02.0-1.el9.x86_64.r 718 kB/s | 36 kB 00:00 2024-05-03T18:34:42.990 INFO:teuthology.orchestra.run.smithi045.stdout:(95/152): qatlib-24.02.0-1.el9.x86_64.rpm 1.8 MB/s | 221 kB 00:00 2024-05-03T18:34:43.040 INFO:teuthology.orchestra.run.smithi045.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 518 kB/s | 47 kB 00:00 2024-05-03T18:34:43.091 INFO:teuthology.orchestra.run.smithi045.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 3.0 MB/s | 305 kB 00:00 2024-05-03T18:34:43.116 INFO:teuthology.orchestra.run.smithi045.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 767 kB/s | 58 kB 00:00 2024-05-03T18:34:43.174 INFO:teuthology.orchestra.run.smithi045.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.1 MB/s | 94 kB 00:00 2024-05-03T18:34:43.200 INFO:teuthology.orchestra.run.smithi045.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 762 kB/s | 64 kB 00:00 2024-05-03T18:34:43.350 INFO:teuthology.orchestra.run.smithi113.stdout:(152/152): ceph-test-19.0.0-3450.g9a047fe7.el9. 3.6 MB/s | 49 MB 00:13 2024-05-03T18:34:43.355 INFO:teuthology.orchestra.run.smithi113.stdout:-------------------------------------------------------------------------------- 2024-05-03T18:34:43.355 INFO:teuthology.orchestra.run.smithi113.stdout:Total 12 MB/s | 222 MB 00:18 2024-05-03T18:34:43.358 INFO:teuthology.orchestra.run.smithi045.stdout:(101/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 28 MB/s | 16 MB 00:00 2024-05-03T18:34:43.383 INFO:teuthology.orchestra.run.smithi045.stdout:(102/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 214 kB/s | 45 kB 00:00 2024-05-03T18:34:43.425 INFO:teuthology.orchestra.run.smithi045.stdout:(103/152): lua-devel-5.4.4-4.el9.x86_64.rpm 99 kB/s | 22 kB 00:00 2024-05-03T18:34:43.567 INFO:teuthology.orchestra.run.smithi045.stdout:(104/152): gperftools-libs-2.9.1-2.el9.x86_64.r 2.1 MB/s | 309 kB 00:00 2024-05-03T18:34:43.602 INFO:teuthology.orchestra.run.smithi045.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 584 kB/s | 19 kB 00:00 2024-05-03T18:34:43.626 INFO:teuthology.orchestra.run.smithi045.stdout:(106/152): protobuf-compiler-3.14.0-13.el9.x86_ 3.2 MB/s | 863 kB 00:00 2024-05-03T18:34:43.660 INFO:teuthology.orchestra.run.smithi045.stdout:(107/152): abseil-cpp-20211102.0-3.el9.x86_64.r 1.9 MB/s | 548 kB 00:00 2024-05-03T18:34:43.718 INFO:teuthology.orchestra.run.smithi045.stdout:(108/152): libarrow-doc-9.0.0-11.el9.noarch.rpm 284 kB/s | 26 kB 00:00 2024-05-03T18:34:43.743 INFO:teuthology.orchestra.run.smithi045.stdout:(109/152): liboath-2.6.7-2.el9.x86_64.rpm 583 kB/s | 49 kB 00:00 2024-05-03T18:34:43.811 INFO:teuthology.orchestra.run.smithi045.stdout:(110/152): libarrow-9.0.0-11.el9.x86_64.rpm 21 MB/s | 4.4 MB 00:00 2024-05-03T18:34:43.836 INFO:teuthology.orchestra.run.smithi045.stdout:(111/152): libunwind-1.6.2-1.el9.x86_64.rpm 575 kB/s | 67 kB 00:00 2024-05-03T18:34:43.886 INFO:teuthology.orchestra.run.smithi045.stdout:(112/152): parquet-libs-9.0.0-11.el9.x86_64.rpm 11 MB/s | 839 kB 00:00 2024-05-03T18:34:43.911 INFO:teuthology.orchestra.run.smithi045.stdout:(113/152): luarocks-3.9.2-1.el9.noarch.rpm 902 kB/s | 151 kB 00:00 2024-05-03T18:34:43.937 INFO:teuthology.orchestra.run.smithi045.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 865 kB/s | 43 kB 00:00 2024-05-03T18:34:43.962 INFO:teuthology.orchestra.run.smithi045.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 641 kB/s | 32 kB 00:00 2024-05-03T18:34:43.996 INFO:teuthology.orchestra.run.smithi045.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 255 kB/s | 15 kB 00:00 2024-05-03T18:34:44.021 INFO:teuthology.orchestra.run.smithi045.stdout:(117/152): python3-cheroot-8.6.0-4.el9.noarch.r 2.9 MB/s | 172 kB 00:00 2024-05-03T18:34:44.063 INFO:teuthology.orchestra.run.smithi045.stdout:(118/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 7.1 MB/s | 303 kB 00:00 2024-05-03T18:34:44.088 INFO:teuthology.orchestra.run.smithi045.stdout:(119/152): python3-asyncssh-2.13.2-5.el9.noarch 2.1 MB/s | 548 kB 00:00 2024-05-03T18:34:44.113 INFO:teuthology.orchestra.run.smithi045.stdout:(120/152): python3-colorama-0.4.4-12.el9.noarch 703 kB/s | 35 kB 00:00 2024-05-03T18:34:44.139 INFO:teuthology.orchestra.run.smithi045.stdout:(121/152): python3-cherrypy-18.6.1-2.el9.noarch 2.5 MB/s | 358 kB 00:00 2024-05-03T18:34:44.197 INFO:teuthology.orchestra.run.smithi045.stdout:(122/152): python3-grpcio-tools-1.46.7-10.el9.x 2.4 MB/s | 144 kB 00:00 2024-05-03T18:34:44.223 INFO:teuthology.orchestra.run.smithi045.stdout:(123/152): python3-google-auth-2.29.0-1.el9.noa 1.5 MB/s | 206 kB 00:00 2024-05-03T18:34:44.248 INFO:teuthology.orchestra.run.smithi045.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 214 kB/s | 11 kB 00:00 2024-05-03T18:34:44.273 INFO:teuthology.orchestra.run.smithi045.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 352 kB/s | 18 kB 00:00 2024-05-03T18:34:44.298 INFO:teuthology.orchestra.run.smithi045.stdout:(126/152): python3-jaraco-collections-3.0.0-8.e 461 kB/s | 23 kB 00:00 2024-05-03T18:34:44.349 INFO:teuthology.orchestra.run.smithi045.stdout:(127/152): python3-grpcio-1.46.7-10.el9.x86_64. 8.7 MB/s | 2.0 MB 00:00 2024-05-03T18:34:44.374 INFO:teuthology.orchestra.run.smithi045.stdout:(128/152): python3-jaraco-functools-3.5.0-2.el9 194 kB/s | 19 kB 00:00 2024-05-03T18:34:44.399 INFO:teuthology.orchestra.run.smithi045.stdout:(129/152): python3-jaraco-text-3.2.0-6.el9.noar 196 kB/s | 20 kB 00:00 2024-05-03T18:34:44.424 INFO:teuthology.orchestra.run.smithi045.stdout:(130/152): python3-logutils-0.3.5-21.el9.noarch 921 kB/s | 46 kB 00:00 2024-05-03T18:34:44.450 INFO:teuthology.orchestra.run.smithi045.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 1.5 MB/s | 79 kB 00:00 2024-05-03T18:34:44.456 INFO:teuthology.orchestra.run.smithi113.stdout:Running transaction check 2024-05-03T18:34:44.475 INFO:teuthology.orchestra.run.smithi045.stdout:(132/152): python3-natsort-7.1.1-5.el9.noarch.r 1.1 MB/s | 58 kB 00:00 2024-05-03T18:34:44.500 INFO:teuthology.orchestra.run.smithi045.stdout:(133/152): python3-pecan-1.4.2-3.el9.noarch.rpm 5.3 MB/s | 272 kB 00:00 2024-05-03T18:34:44.526 INFO:teuthology.orchestra.run.smithi045.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 328 kB/s | 16 kB 00:00 2024-05-03T18:34:44.550 INFO:teuthology.orchestra.run.smithi045.stdout:(135/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.8 MB/s | 90 kB 00:00 2024-05-03T18:34:44.576 INFO:teuthology.orchestra.run.smithi045.stdout:(136/152): python3-pyperclip-1.8.2-2.el9.noarch 563 kB/s | 28 kB 00:00 2024-05-03T18:34:44.599 INFO:teuthology.orchestra.run.smithi113.stdout:Transaction check succeeded. 2024-05-03T18:34:44.599 INFO:teuthology.orchestra.run.smithi113.stdout:Running transaction test 2024-05-03T18:34:44.601 INFO:teuthology.orchestra.run.smithi045.stdout:(137/152): python3-repoze-lru-0.7-16.el9.noarch 613 kB/s | 31 kB 00:00 2024-05-03T18:34:44.626 INFO:teuthology.orchestra.run.smithi045.stdout:(138/152): python3-routes-2.5.1-5.el9.noarch.rp 3.7 MB/s | 188 kB 00:00 2024-05-03T18:34:44.677 INFO:teuthology.orchestra.run.smithi045.stdout:(139/152): python3-rsa-4.9-2.el9.noarch.rpm 785 kB/s | 59 kB 00:00 2024-05-03T18:34:44.702 INFO:teuthology.orchestra.run.smithi045.stdout:(140/152): python3-tempora-5.0.0-2.el9.noarch.r 475 kB/s | 36 kB 00:00 2024-05-03T18:34:44.727 INFO:teuthology.orchestra.run.smithi045.stdout:(141/152): python3-typing-extensions-4.4.0-2.el 1.0 MB/s | 51 kB 00:00 2024-05-03T18:34:44.761 INFO:teuthology.orchestra.run.smithi045.stdout:(142/152): python3-kubernetes-26.1.0-2.el9.noar 2.5 MB/s | 1.0 MB 00:00 2024-05-03T18:34:44.786 INFO:teuthology.orchestra.run.smithi045.stdout:(143/152): python3-webob-1.8.7-6.el9.noarch.rpm 2.7 MB/s | 230 kB 00:00 2024-05-03T18:34:44.812 INFO:teuthology.orchestra.run.smithi045.stdout:(144/152): python3-websocket-client-1.2.3-2.el9 1.0 MB/s | 90 kB 00:00 2024-05-03T18:34:44.837 INFO:teuthology.orchestra.run.smithi045.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 399 kB/s | 20 kB 00:00 2024-05-03T18:34:44.862 INFO:teuthology.orchestra.run.smithi045.stdout:(146/152): re2-20211101-3.el9.x86_64.rpm 3.7 MB/s | 192 kB 00:00 2024-05-03T18:34:44.913 INFO:teuthology.orchestra.run.smithi045.stdout:(147/152): thrift-0.15.0-2.el9.x86_64.rpm 21 MB/s | 1.6 MB 00:00 2024-05-03T18:34:44.938 INFO:teuthology.orchestra.run.smithi045.stdout:(148/152): python3-werkzeug-2.0.3-3.el9.1.noarc 2.4 MB/s | 427 kB 00:00 2024-05-03T18:34:45.122 INFO:teuthology.orchestra.run.smithi045.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 7.4 MB/s | 1.3 MB 00:00 2024-05-03T18:34:45.247 INFO:teuthology.orchestra.run.smithi045.stdout:(150/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 19 MB/s | 2.4 MB 00:00 2024-05-03T18:34:45.764 INFO:teuthology.orchestra.run.smithi045.stdout:(151/152): librbd1-19.0.0-3450.g9a047fe7.el9.x8 3.7 MB/s | 3.1 MB 00:00 2024-05-03T18:34:45.813 INFO:teuthology.orchestra.run.smithi045.stdout:(152/152): librados2-19.0.0-3450.g9a047fe7.el9. 3.6 MB/s | 3.4 MB 00:00 2024-05-03T18:34:45.817 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2024-05-03T18:34:45.818 INFO:teuthology.orchestra.run.smithi045.stdout:Total 10 MB/s | 222 MB 00:21 2024-05-03T18:34:46.407 INFO:teuthology.orchestra.run.smithi113.stdout:Transaction test succeeded. 2024-05-03T18:34:46.407 INFO:teuthology.orchestra.run.smithi113.stdout:Running transaction 2024-05-03T18:34:46.932 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction check 2024-05-03T18:34:47.094 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction check succeeded. 2024-05-03T18:34:47.094 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction test 2024-05-03T18:34:48.978 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction test succeeded. 2024-05-03T18:34:48.978 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction 2024-05-03T18:34:49.123 INFO:teuthology.orchestra.run.smithi032.stdout: Preparing : 1/1 2024-05-03T18:34:49.203 INFO:teuthology.orchestra.run.smithi032.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-05-03T18:34:49.241 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-05-03T18:34:49.506 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-05-03T18:34:49.547 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-05-03T18:34:49.571 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-05-03T18:34:49.616 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-05-03T18:34:49.962 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-05-03T18:34:50.068 INFO:teuthology.orchestra.run.smithi032.stdout: Upgrading : librados2-2:19.0.0-3450.g9a047fe7.el9.x86_64 8/156 2024-05-03T18:34:50.197 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: librados2-2:19.0.0-3450.g9a047fe7.el9.x86_64 8/156 2024-05-03T18:34:50.223 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : libcephfs2-2:19.0.0-3450.g9a047fe7.el9.x86_64 9/156 2024-05-03T18:34:50.299 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: libcephfs2-2:19.0.0-3450.g9a047fe7.el9.x86_64 9/156 2024-05-03T18:34:50.336 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-rados-2:19.0.0-3450.g9a047fe7.el9.x86_64 10/156 2024-05-03T18:34:50.493 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-05-03T18:34:50.642 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-05-03T18:34:50.670 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : libcephsqlite-2:19.0.0-3450.g9a047fe7.el9.x86_64 13/156 2024-05-03T18:34:50.775 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3450.g9a047fe7.el9.x86_64 13/156 2024-05-03T18:34:50.804 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : libradosstriper1-2:19.0.0-3450.g9a047fe7.el9.x86 14/156 2024-05-03T18:34:50.867 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3450.g9a047fe7.el9.x86 14/156 2024-05-03T18:34:50.888 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-05-03T18:34:50.911 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-05-03T18:34:51.080 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-05-03T18:34:51.113 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-05-03T18:34:51.225 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-05-03T18:34:51.321 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 18/156 2024-05-03T18:34:51.371 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-05-03T18:34:51.407 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-05-03T18:34:51.483 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-05-03T18:34:51.522 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-05-03T18:34:51.567 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-05-03T18:34:51.615 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-05-03T18:34:51.643 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-05-03T18:34:51.714 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-05-03T18:34:51.775 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-05-03T18:34:51.804 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-05-03T18:34:51.839 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-05-03T18:34:51.919 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-05-03T18:34:51.950 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-ceph-common-2:19.0.0-3450.g9a047fe7.el9. 31/156 2024-05-03T18:34:51.994 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-ceph-argparse-2:19.0.0-3450.g9a047fe7.el 32/156 2024-05-03T18:34:52.022 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-cephfs-2:19.0.0-3450.g9a047fe7.el9.x86_6 33/156 2024-05-03T18:34:52.094 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-05-03T18:34:52.137 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-05-03T18:34:52.215 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-05-03T18:34:52.361 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-05-03T18:34:52.385 INFO:teuthology.orchestra.run.smithi113.stdout: Preparing : 1/1 2024-05-03T18:34:52.415 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-05-03T18:34:52.495 INFO:teuthology.orchestra.run.smithi113.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-05-03T18:34:52.534 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-05-03T18:34:52.680 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-05-03T18:34:52.847 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-05-03T18:34:52.872 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-05-03T18:34:52.916 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-05-03T18:34:52.981 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-05-03T18:34:53.072 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-05-03T18:34:53.106 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-05-03T18:34:53.149 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-05-03T18:34:53.187 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : librados-devel-2:19.0.0-3450.g9a047fe7.el9.x86_6 43/156 2024-05-03T18:34:53.229 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-05-03T18:34:53.252 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-05-03T18:34:53.365 INFO:teuthology.orchestra.run.smithi113.stdout: Upgrading : librados2-2:19.0.0-3450.g9a047fe7.el9.x86_64 8/156 2024-05-03T18:34:53.382 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-05-03T18:34:53.440 INFO:teuthology.orchestra.run.smithi032.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-05-03T18:34:53.485 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: librados2-2:19.0.0-3450.g9a047fe7.el9.x86_64 8/156 2024-05-03T18:34:53.496 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-05-03T18:34:53.524 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : libcephfs2-2:19.0.0-3450.g9a047fe7.el9.x86_64 9/156 2024-05-03T18:34:53.570 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-05-03T18:34:53.607 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-05-03T18:34:53.614 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: libcephfs2-2:19.0.0-3450.g9a047fe7.el9.x86_64 9/156 2024-05-03T18:34:53.644 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-05-03T18:34:53.645 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-rados-2:19.0.0-3450.g9a047fe7.el9.x86_64 10/156 2024-05-03T18:34:53.691 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-05-03T18:34:53.727 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-05-03T18:34:53.770 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-05-03T18:34:53.798 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-05-03T18:34:53.812 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-05-03T18:34:53.842 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-05-03T18:34:53.870 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : libcephsqlite-2:19.0.0-3450.g9a047fe7.el9.x86_64 13/156 2024-05-03T18:34:53.956 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-05-03T18:34:53.975 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3450.g9a047fe7.el9.x86_64 13/156 2024-05-03T18:34:53.996 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : libradosstriper1-2:19.0.0-3450.g9a047fe7.el9.x86 14/156 2024-05-03T18:34:54.001 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-05-03T18:34:54.031 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-05-03T18:34:54.059 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3450.g9a047fe7.el9.x86 14/156 2024-05-03T18:34:54.073 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-05-03T18:34:54.081 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-05-03T18:34:54.106 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-05-03T18:34:54.173 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-05-03T18:34:54.206 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-05-03T18:34:54.246 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-05-03T18:34:54.271 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-05-03T18:34:54.296 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 63/156 2024-05-03T18:34:54.305 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-05-03T18:34:54.338 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-05-03T18:34:54.376 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-05-03T18:34:54.446 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-05-03T18:34:54.512 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-05-03T18:34:54.534 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 18/156 2024-05-03T18:34:54.563 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-05-03T18:34:54.598 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-05-03T18:34:54.637 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-05-03T18:34:54.666 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-05-03T18:34:54.688 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-05-03T18:34:54.700 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-05-03T18:34:54.730 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-05-03T18:34:54.740 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-05-03T18:34:54.774 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-05-03T18:34:54.779 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-05-03T18:34:54.806 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-05-03T18:34:54.809 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-05-03T18:34:54.843 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-05-03T18:34:54.843 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-05-03T18:34:54.876 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-05-03T18:34:54.919 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-05-03T18:34:54.920 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-05-03T18:34:54.953 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-05-03T18:34:54.974 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-05-03T18:34:55.004 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-05-03T18:34:55.039 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-05-03T18:34:55.044 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-05-03T18:34:55.092 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-05-03T18:34:55.102 INFO:teuthology.orchestra.run.smithi045.stdout: Preparing : 1/1 2024-05-03T18:34:55.116 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-05-03T18:34:55.123 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-05-03T18:34:55.150 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-ceph-common-2:19.0.0-3450.g9a047fe7.el9. 31/156 2024-05-03T18:34:55.171 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-05-03T18:34:55.184 INFO:teuthology.orchestra.run.smithi045.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-05-03T18:34:55.193 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-ceph-argparse-2:19.0.0-3450.g9a047fe7.el 32/156 2024-05-03T18:34:55.222 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-cephfs-2:19.0.0-3450.g9a047fe7.el9.x86_6 33/156 2024-05-03T18:34:55.224 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-05-03T18:34:55.283 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-05-03T18:34:55.293 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-05-03T18:34:55.333 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-05-03T18:34:55.345 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-05-03T18:34:55.362 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-05-03T18:34:55.421 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-05-03T18:34:55.520 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-05-03T18:34:55.544 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-05-03T18:34:55.556 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-05-03T18:34:55.589 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-05-03T18:34:55.607 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-05-03T18:34:55.933 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-05-03T18:34:56.045 INFO:teuthology.orchestra.run.smithi045.stdout: Upgrading : librados2-2:19.0.0-3450.g9a047fe7.el9.x86_64 8/156 2024-05-03T18:34:56.101 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-05-03T18:34:56.137 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-05-03T18:34:56.155 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-05-03T18:34:56.169 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: librados2-2:19.0.0-3450.g9a047fe7.el9.x86_64 8/156 2024-05-03T18:34:56.181 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-05-03T18:34:56.196 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libcephfs2-2:19.0.0-3450.g9a047fe7.el9.x86_64 9/156 2024-05-03T18:34:56.219 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-05-03T18:34:56.248 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-05-03T18:34:56.283 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-05-03T18:34:56.288 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: libcephfs2-2:19.0.0-3450.g9a047fe7.el9.x86_64 9/156 2024-05-03T18:34:56.291 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-05-03T18:34:56.325 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-rados-2:19.0.0-3450.g9a047fe7.el9.x86_64 10/156 2024-05-03T18:34:56.328 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : librados-devel-2:19.0.0-3450.g9a047fe7.el9.x86_6 43/156 2024-05-03T18:34:56.339 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-05-03T18:34:56.371 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-05-03T18:34:56.429 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-05-03T18:34:56.466 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-05-03T18:34:56.490 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-05-03T18:34:56.528 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-05-03T18:34:56.539 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-05-03T18:34:56.566 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libcephsqlite-2:19.0.0-3450.g9a047fe7.el9.x86_64 13/156 2024-05-03T18:34:56.590 INFO:teuthology.orchestra.run.smithi113.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-05-03T18:34:56.645 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-05-03T18:34:56.650 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-05-03T18:34:56.655 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3450.g9a047fe7.el9.x86_64 13/156 2024-05-03T18:34:56.675 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libradosstriper1-2:19.0.0-3450.g9a047fe7.el9.x86 14/156 2024-05-03T18:34:56.680 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-05-03T18:34:56.719 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-05-03T18:34:56.738 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3450.g9a047fe7.el9.x86 14/156 2024-05-03T18:34:56.749 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-05-03T18:34:56.760 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-05-03T18:34:56.760 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-05-03T18:34:56.779 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-05-03T18:34:56.785 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-05-03T18:34:56.832 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-05-03T18:34:56.868 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-05-03T18:34:56.912 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-05-03T18:34:56.954 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-05-03T18:34:56.979 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-05-03T18:34:57.010 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-05-03T18:34:57.090 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-05-03T18:34:57.125 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-05-03T18:34:57.134 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-05-03T18:34:57.164 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-05-03T18:34:57.206 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-05-03T18:34:57.226 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 18/156 2024-05-03T18:34:57.259 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-05-03T18:34:57.295 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-05-03T18:34:57.301 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-05-03T18:34:57.339 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-05-03T18:34:57.370 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-05-03T18:34:57.379 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-05-03T18:34:57.405 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-05-03T18:34:57.418 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-05-03T18:34:57.430 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 63/156 2024-05-03T18:34:57.439 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-05-03T18:34:57.462 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-05-03T18:34:57.494 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-05-03T18:34:57.505 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-05-03T18:34:57.522 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-05-03T18:34:57.549 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-05-03T18:34:57.594 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-05-03T18:34:57.632 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-05-03T18:34:57.654 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-05-03T18:34:57.671 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-05-03T18:34:57.683 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-05-03T18:34:57.699 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-05-03T18:34:57.733 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-05-03T18:34:57.736 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-05-03T18:34:57.773 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-05-03T18:34:57.804 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-05-03T18:34:57.814 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-05-03T18:34:57.834 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-05-03T18:34:57.846 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-ceph-common-2:19.0.0-3450.g9a047fe7.el9. 31/156 2024-05-03T18:34:57.868 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-05-03T18:34:57.890 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-ceph-argparse-2:19.0.0-3450.g9a047fe7.el 32/156 2024-05-03T18:34:57.903 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-05-03T18:34:57.918 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-cephfs-2:19.0.0-3450.g9a047fe7.el9.x86_6 33/156 2024-05-03T18:34:57.943 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-05-03T18:34:57.978 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-05-03T18:34:57.990 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-05-03T18:34:58.033 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-05-03T18:34:58.065 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-05-03T18:34:58.108 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-05-03T18:34:58.118 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-05-03T18:34:58.139 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-05-03T18:34:58.195 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-05-03T18:34:58.264 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-05-03T18:34:58.293 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-05-03T18:34:58.319 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-05-03T18:34:58.350 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-05-03T18:34:58.842 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-05-03T18:34:58.886 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-05-03T18:34:58.910 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-05-03T18:34:58.939 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-05-03T18:34:58.972 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-05-03T18:34:58.975 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-05-03T18:34:59.017 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-05-03T18:34:59.060 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-05-03T18:34:59.084 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-05-03T18:34:59.098 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : librados-devel-2:19.0.0-3450.g9a047fe7.el9.x86_6 43/156 2024-05-03T18:34:59.130 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-05-03T18:34:59.140 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-05-03T18:34:59.164 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-05-03T18:34:59.264 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-05-03T18:34:59.301 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-05-03T18:34:59.322 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-05-03T18:34:59.334 INFO:teuthology.orchestra.run.smithi032.stdout: Upgrading : librbd1-2:19.0.0-3450.g9a047fe7.el9.x86_64 99/156 2024-05-03T18:34:59.338 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-05-03T18:34:59.401 INFO:teuthology.orchestra.run.smithi045.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-05-03T18:34:59.411 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-05-03T18:34:59.449 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-05-03T18:34:59.451 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: librbd1-2:19.0.0-3450.g9a047fe7.el9.x86_64 99/156 2024-05-03T18:34:59.457 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-05-03T18:34:59.481 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-rbd-2:19.0.0-3450.g9a047fe7.el9.x86_64 100/156 2024-05-03T18:34:59.518 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-05-03T18:34:59.532 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-05-03T18:34:59.576 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-05-03T18:34:59.613 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-05-03T18:34:59.657 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-05-03T18:34:59.661 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-05-03T18:34:59.695 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-05-03T18:34:59.739 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-05-03T18:34:59.782 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-05-03T18:34:59.814 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-05-03T18:34:59.892 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-05-03T18:34:59.926 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-05-03T18:34:59.970 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-05-03T18:34:59.991 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-05-03T18:35:00.027 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 102/156 2024-05-03T18:35:00.033 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-05-03T18:35:00.055 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : librgw2-2:19.0.0-3450.g9a047fe7.el9.x86_64 103/156 2024-05-03T18:35:00.126 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: librgw2-2:19.0.0-3450.g9a047fe7.el9.x86_64 103/156 2024-05-03T18:35:00.132 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-05-03T18:35:00.156 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-rgw-2:19.0.0-3450.g9a047fe7.el9.x86_64 104/156 2024-05-03T18:35:00.166 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-05-03T18:35:00.206 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-05-03T18:35:00.240 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-05-03T18:35:00.264 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 63/156 2024-05-03T18:35:00.345 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-05-03T18:35:00.482 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-05-03T18:35:00.513 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-05-03T18:35:00.540 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-05-03T18:35:00.541 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-05-03T18:35:00.575 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-05-03T18:35:00.615 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-05-03T18:35:00.654 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-05-03T18:35:00.655 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-05-03T18:35:00.684 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-05-03T18:35:00.719 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-05-03T18:35:00.752 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-05-03T18:35:00.794 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-05-03T18:35:00.828 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-05-03T18:35:00.920 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-05-03T18:35:00.967 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-05-03T18:35:01.006 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-05-03T18:35:01.054 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-05-03T18:35:01.167 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-05-03T18:35:01.226 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-05-03T18:35:01.886 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-05-03T18:35:01.976 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-05-03T18:35:02.005 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-05-03T18:35:02.016 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-05-03T18:35:02.038 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-05-03T18:35:02.063 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-05-03T18:35:02.096 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-05-03T18:35:02.196 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-05-03T18:35:02.247 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-05-03T18:35:02.339 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-05-03T18:35:02.349 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-05-03T18:35:02.373 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-05-03T18:35:02.403 INFO:teuthology.orchestra.run.smithi113.stdout: Upgrading : librbd1-2:19.0.0-3450.g9a047fe7.el9.x86_64 99/156 2024-05-03T18:35:02.476 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: librbd1-2:19.0.0-3450.g9a047fe7.el9.x86_64 99/156 2024-05-03T18:35:02.508 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-rbd-2:19.0.0-3450.g9a047fe7.el9.x86_64 100/156 2024-05-03T18:35:02.543 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-common-2:19.0.0-3450.g9a047fe7.el9.x86_64 105/156 2024-05-03T18:35:02.544 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-05-03T18:35:02.560 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-05-03T18:35:02.578 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-common-2:19.0.0-3450.g9a047fe7.el9.x86_64 105/156 2024-05-03T18:35:02.595 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-05-03T18:35:02.676 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-05-03T18:35:03.022 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 102/156 2024-05-03T18:35:03.048 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : librgw2-2:19.0.0-3450.g9a047fe7.el9.x86_64 103/156 2024-05-03T18:35:03.119 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: librgw2-2:19.0.0-3450.g9a047fe7.el9.x86_64 103/156 2024-05-03T18:35:03.141 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-rgw-2:19.0.0-3450.g9a047fe7.el9.x86_64 104/156 2024-05-03T18:35:03.149 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-common-2:19.0.0-3450.g9a047fe7.el9.x86_64 105/156 2024-05-03T18:35:03.190 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-base-2:19.0.0-3450.g9a047fe7.el9.x86_64 106/156 2024-05-03T18:35:03.283 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-base-2:19.0.0-3450.g9a047fe7.el9.x86_64 106/156 2024-05-03T18:35:03.283 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-03T18:35:03.283 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-03T18:35:03.283 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-03T18:35:03.316 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-selinux-2:19.0.0-3450.g9a047fe7.el9.x86_64 107/156 2024-05-03T18:35:03.376 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-05-03T18:35:03.499 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-05-03T18:35:04.824 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-05-03T18:35:04.906 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-05-03T18:35:04.934 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-05-03T18:35:04.967 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-05-03T18:35:05.297 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-05-03T18:35:05.338 INFO:teuthology.orchestra.run.smithi045.stdout: Upgrading : librbd1-2:19.0.0-3450.g9a047fe7.el9.x86_64 99/156 2024-05-03T18:35:05.426 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-common-2:19.0.0-3450.g9a047fe7.el9.x86_64 105/156 2024-05-03T18:35:05.430 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: librbd1-2:19.0.0-3450.g9a047fe7.el9.x86_64 99/156 2024-05-03T18:35:05.460 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-rbd-2:19.0.0-3450.g9a047fe7.el9.x86_64 100/156 2024-05-03T18:35:05.462 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-common-2:19.0.0-3450.g9a047fe7.el9.x86_64 105/156 2024-05-03T18:35:05.496 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-05-03T18:35:06.004 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-common-2:19.0.0-3450.g9a047fe7.el9.x86_64 105/156 2024-05-03T18:35:06.005 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 102/156 2024-05-03T18:35:06.032 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : librgw2-2:19.0.0-3450.g9a047fe7.el9.x86_64 103/156 2024-05-03T18:35:06.049 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-base-2:19.0.0-3450.g9a047fe7.el9.x86_64 106/156 2024-05-03T18:35:06.104 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: librgw2-2:19.0.0-3450.g9a047fe7.el9.x86_64 103/156 2024-05-03T18:35:06.140 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-base-2:19.0.0-3450.g9a047fe7.el9.x86_64 106/156 2024-05-03T18:35:06.141 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-03T18:35:06.141 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-03T18:35:06.141 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-05-03T18:35:06.159 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-rgw-2:19.0.0-3450.g9a047fe7.el9.x86_64 104/156 2024-05-03T18:35:06.167 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-selinux-2:19.0.0-3450.g9a047fe7.el9.x86_64 107/156 2024-05-03T18:35:08.578 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-common-2:19.0.0-3450.g9a047fe7.el9.x86_64 105/156 2024-05-03T18:35:08.613 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-common-2:19.0.0-3450.g9a047fe7.el9.x86_64 105/156 2024-05-03T18:35:09.187 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-common-2:19.0.0-3450.g9a047fe7.el9.x86_64 105/156 2024-05-03T18:35:09.233 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-base-2:19.0.0-3450.g9a047fe7.el9.x86_64 106/156 2024-05-03T18:35:09.310 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-base-2:19.0.0-3450.g9a047fe7.el9.x86_64 106/156 2024-05-03T18:35:09.310 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-03T18:35:09.310 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-03T18:35:09.310 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-03T18:35:09.359 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-selinux-2:19.0.0-3450.g9a047fe7.el9.x86_64 107/156 2024-05-03T18:35:15.155 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3450.g9a047fe7.el9.x86_64 107/156 2024-05-03T18:35:15.156 INFO:teuthology.orchestra.run.smithi032.stdout:skipping the directory /sys 2024-05-03T18:35:15.156 INFO:teuthology.orchestra.run.smithi032.stdout:skipping the directory /proc 2024-05-03T18:35:15.156 INFO:teuthology.orchestra.run.smithi032.stdout:skipping the directory /mnt 2024-05-03T18:35:15.156 INFO:teuthology.orchestra.run.smithi032.stdout:skipping the directory /var/tmp 2024-05-03T18:35:15.156 INFO:teuthology.orchestra.run.smithi032.stdout:skipping the directory /home 2024-05-03T18:35:15.156 INFO:teuthology.orchestra.run.smithi032.stdout:skipping the directory /root 2024-05-03T18:35:15.156 INFO:teuthology.orchestra.run.smithi032.stdout:skipping the directory /tmp 2024-05-03T18:35:15.156 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-03T18:35:15.326 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-volume-2:19.0.0-3450.g9a047fe7.el9.noarch 108/156 2024-05-03T18:35:15.345 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-volume-2:19.0.0-3450.g9a047fe7.el9.noarch 108/156 2024-05-03T18:35:15.346 INFO:teuthology.orchestra.run.smithi032.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-03T18:35:15.346 INFO:teuthology.orchestra.run.smithi032.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-03T18:35:15.346 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-03T18:35:16.880 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-osd-2:19.0.0-3450.g9a047fe7.el9.x86_64 109/156 2024-05-03T18:35:16.906 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-osd-2:19.0.0-3450.g9a047fe7.el9.x86_64 109/156 2024-05-03T18:35:16.906 INFO:teuthology.orchestra.run.smithi032.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-03T18:35:16.907 INFO:teuthology.orchestra.run.smithi032.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-03T18:35:16.907 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-03T18:35:16.907 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-03T18:35:16.907 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-03T18:35:17.138 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-mds-2:19.0.0-3450.g9a047fe7.el9.x86_64 110/156 2024-05-03T18:35:17.162 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-mds-2:19.0.0-3450.g9a047fe7.el9.x86_64 110/156 2024-05-03T18:35:17.163 INFO:teuthology.orchestra.run.smithi032.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-03T18:35:17.163 INFO:teuthology.orchestra.run.smithi032.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-03T18:35:17.163 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-03T18:35:17.163 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-03T18:35:17.163 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-03T18:35:17.631 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-mon-2:19.0.0-3450.g9a047fe7.el9.x86_64 111/156 2024-05-03T18:35:17.654 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-mon-2:19.0.0-3450.g9a047fe7.el9.x86_64 111/156 2024-05-03T18:35:17.654 INFO:teuthology.orchestra.run.smithi032.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-03T18:35:17.655 INFO:teuthology.orchestra.run.smithi032.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-03T18:35:17.655 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-03T18:35:17.655 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-03T18:35:17.655 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-03T18:35:17.761 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-05-03T18:35:17.820 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-05-03T18:35:17.863 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3450.g9a047fe7.el9.x86_64 107/156 2024-05-03T18:35:17.864 INFO:teuthology.orchestra.run.smithi113.stdout:skipping the directory /sys 2024-05-03T18:35:17.864 INFO:teuthology.orchestra.run.smithi113.stdout:skipping the directory /proc 2024-05-03T18:35:17.864 INFO:teuthology.orchestra.run.smithi113.stdout:skipping the directory /mnt 2024-05-03T18:35:17.864 INFO:teuthology.orchestra.run.smithi113.stdout:skipping the directory /var/tmp 2024-05-03T18:35:17.864 INFO:teuthology.orchestra.run.smithi113.stdout:skipping the directory /home 2024-05-03T18:35:17.864 INFO:teuthology.orchestra.run.smithi113.stdout:skipping the directory /root 2024-05-03T18:35:17.864 INFO:teuthology.orchestra.run.smithi113.stdout:skipping the directory /tmp 2024-05-03T18:35:17.864 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-05-03T18:35:17.884 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-05-03T18:35:18.025 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-volume-2:19.0.0-3450.g9a047fe7.el9.noarch 108/156 2024-05-03T18:35:18.040 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-volume-2:19.0.0-3450.g9a047fe7.el9.noarch 108/156 2024-05-03T18:35:18.040 INFO:teuthology.orchestra.run.smithi113.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-03T18:35:18.040 INFO:teuthology.orchestra.run.smithi113.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-03T18:35:18.040 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-05-03T18:35:18.081 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-05-03T18:35:18.132 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-05-03T18:35:18.212 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-05-03T18:35:18.302 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-05-03T18:35:18.447 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-05-03T18:35:18.499 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-05-03T18:35:18.530 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-05-03T18:35:18.558 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-05-03T18:35:18.920 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: cephadm-2:19.0.0-3450.g9a047fe7.el9.noarch 123/156 2024-05-03T18:35:18.948 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : cephadm-2:19.0.0-3450.g9a047fe7.el9.noarch 123/156 2024-05-03T18:35:19.011 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3450.g9a047fe7.e 124/156 2024-05-03T18:35:19.078 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3450.g9a047fe7. 125/156 2024-05-03T18:35:19.104 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3450.g9a047fe7.el9.noa 126/156 2024-05-03T18:35:19.519 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-osd-2:19.0.0-3450.g9a047fe7.el9.x86_64 109/156 2024-05-03T18:35:19.545 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-osd-2:19.0.0-3450.g9a047fe7.el9.x86_64 109/156 2024-05-03T18:35:19.545 INFO:teuthology.orchestra.run.smithi113.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-03T18:35:19.545 INFO:teuthology.orchestra.run.smithi113.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-03T18:35:19.545 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-03T18:35:19.545 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-03T18:35:19.545 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-05-03T18:35:19.796 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-mds-2:19.0.0-3450.g9a047fe7.el9.x86_64 110/156 2024-05-03T18:35:19.819 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-mds-2:19.0.0-3450.g9a047fe7.el9.x86_64 110/156 2024-05-03T18:35:19.819 INFO:teuthology.orchestra.run.smithi113.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-03T18:35:19.819 INFO:teuthology.orchestra.run.smithi113.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-03T18:35:19.819 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-03T18:35:19.819 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-03T18:35:19.819 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-05-03T18:35:20.198 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3450.g9a047fe7.el9.noa 126/156 2024-05-03T18:35:20.231 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3450.g9a047fe7.el9.n 127/156 2024-05-03T18:35:20.273 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-mon-2:19.0.0-3450.g9a047fe7.el9.x86_64 111/156 2024-05-03T18:35:20.295 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-mon-2:19.0.0-3450.g9a047fe7.el9.x86_64 111/156 2024-05-03T18:35:20.295 INFO:teuthology.orchestra.run.smithi113.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-03T18:35:20.296 INFO:teuthology.orchestra.run.smithi113.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-03T18:35:20.296 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-03T18:35:20.296 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-03T18:35:20.296 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-05-03T18:35:20.395 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-05-03T18:35:20.452 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-05-03T18:35:20.516 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-05-03T18:35:20.711 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-05-03T18:35:20.758 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-05-03T18:35:20.835 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-05-03T18:35:20.924 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-05-03T18:35:21.064 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-05-03T18:35:21.081 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3450.g9a047fe7.el9.x86_64 107/156 2024-05-03T18:35:21.082 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /sys 2024-05-03T18:35:21.082 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /proc 2024-05-03T18:35:21.082 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /mnt 2024-05-03T18:35:21.082 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /var/tmp 2024-05-03T18:35:21.082 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /home 2024-05-03T18:35:21.082 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /root 2024-05-03T18:35:21.082 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /tmp 2024-05-03T18:35:21.082 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-03T18:35:21.108 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-05-03T18:35:21.156 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-05-03T18:35:21.184 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-05-03T18:35:21.243 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3450.g9a047fe7.el9.n 127/156 2024-05-03T18:35:21.258 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-volume-2:19.0.0-3450.g9a047fe7.el9.noarch 108/156 2024-05-03T18:35:21.273 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3450.g9a0 128/156 2024-05-03T18:35:21.275 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-volume-2:19.0.0-3450.g9a047fe7.el9.noarch 108/156 2024-05-03T18:35:21.275 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-03T18:35:21.275 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-03T18:35:21.275 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-03T18:35:21.390 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3450.g9a0 128/156 2024-05-03T18:35:21.558 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3450.g9a047fe7.el 129/156 2024-05-03T18:35:21.592 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-mgr-2:19.0.0-3450.g9a047fe7.el9.x86_64 130/156 2024-05-03T18:35:21.617 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3450.g9a047fe7.el9.x86_64 130/156 2024-05-03T18:35:21.617 INFO:teuthology.orchestra.run.smithi032.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-03T18:35:21.617 INFO:teuthology.orchestra.run.smithi032.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-03T18:35:21.617 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-03T18:35:21.617 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-03T18:35:21.617 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-03T18:35:21.636 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: cephadm-2:19.0.0-3450.g9a047fe7.el9.noarch 123/156 2024-05-03T18:35:21.667 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : cephadm-2:19.0.0-3450.g9a047fe7.el9.noarch 123/156 2024-05-03T18:35:21.668 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-mgr-rook-2:19.0.0-3450.g9a047fe7.el9.noarch 131/156 2024-05-03T18:35:21.679 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3450.g9a047fe7.el9.noarch 131/156 2024-05-03T18:35:21.699 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3450.g9a047fe7.e 124/156 2024-05-03T18:35:21.769 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3450.g9a047fe7. 125/156 2024-05-03T18:35:21.780 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-2:19.0.0-3450.g9a047fe7.el9.x86_64 132/156 2024-05-03T18:35:21.806 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3450.g9a047fe7.el9.noa 126/156 2024-05-03T18:35:21.834 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-fuse-2:19.0.0-3450.g9a047fe7.el9.x86_64 133/156 2024-05-03T18:35:22.807 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-05-03T18:35:22.819 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-osd-2:19.0.0-3450.g9a047fe7.el9.x86_64 109/156 2024-05-03T18:35:22.837 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-radosgw-2:19.0.0-3450.g9a047fe7.el9.x86_64 135/156 2024-05-03T18:35:22.840 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3450.g9a047fe7.el9.noa 126/156 2024-05-03T18:35:22.846 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-osd-2:19.0.0-3450.g9a047fe7.el9.x86_64 109/156 2024-05-03T18:35:22.846 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-03T18:35:22.847 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-03T18:35:22.847 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-03T18:35:22.847 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-03T18:35:22.847 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-03T18:35:22.860 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3450.g9a047fe7.el9.x86_64 135/156 2024-05-03T18:35:22.860 INFO:teuthology.orchestra.run.smithi032.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-03T18:35:22.860 INFO:teuthology.orchestra.run.smithi032.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-03T18:35:22.861 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-03T18:35:22.861 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-03T18:35:22.861 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-03T18:35:22.875 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3450.g9a047fe7.el9.n 127/156 2024-05-03T18:35:22.913 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3450.g9a047 136/156 2024-05-03T18:35:22.936 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3450.g9a047 136/156 2024-05-03T18:35:22.936 INFO:teuthology.orchestra.run.smithi032.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-03T18:35:22.936 INFO:teuthology.orchestra.run.smithi032.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-03T18:35:22.936 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-03T18:35:22.988 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : cephfs-mirror-2:19.0.0-3450.g9a047fe7.el9.x86_64 137/156 2024-05-03T18:35:23.011 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3450.g9a047fe7.el9.x86_64 137/156 2024-05-03T18:35:23.012 INFO:teuthology.orchestra.run.smithi032.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-03T18:35:23.012 INFO:teuthology.orchestra.run.smithi032.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-03T18:35:23.012 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-03T18:35:23.012 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-03T18:35:23.012 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-03T18:35:23.087 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mds-2:19.0.0-3450.g9a047fe7.el9.x86_64 110/156 2024-05-03T18:35:23.111 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mds-2:19.0.0-3450.g9a047fe7.el9.x86_64 110/156 2024-05-03T18:35:23.111 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-03T18:35:23.111 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-03T18:35:23.111 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-03T18:35:23.111 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-03T18:35:23.111 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-03T18:35:23.341 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : rbd-mirror-2:19.0.0-3450.g9a047fe7.el9.x86_64 138/156 2024-05-03T18:35:23.365 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3450.g9a047fe7.el9.x86_64 138/156 2024-05-03T18:35:23.365 INFO:teuthology.orchestra.run.smithi032.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-03T18:35:23.365 INFO:teuthology.orchestra.run.smithi032.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-03T18:35:23.365 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-03T18:35:23.365 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-03T18:35:23.365 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-03T18:35:23.579 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mon-2:19.0.0-3450.g9a047fe7.el9.x86_64 111/156 2024-05-03T18:35:23.603 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mon-2:19.0.0-3450.g9a047fe7.el9.x86_64 111/156 2024-05-03T18:35:23.603 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-03T18:35:23.603 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-03T18:35:23.603 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-03T18:35:23.603 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-03T18:35:23.603 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-03T18:35:23.710 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-05-03T18:35:23.768 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-05-03T18:35:23.832 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-05-03T18:35:23.833 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3450.g9a047fe7.el9.n 127/156 2024-05-03T18:35:23.859 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3450.g9a0 128/156 2024-05-03T18:35:23.969 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3450.g9a0 128/156 2024-05-03T18:35:24.030 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-05-03T18:35:24.072 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-05-03T18:35:24.128 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3450.g9a047fe7.el 129/156 2024-05-03T18:35:24.154 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-05-03T18:35:24.161 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-mgr-2:19.0.0-3450.g9a047fe7.el9.x86_64 130/156 2024-05-03T18:35:24.184 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3450.g9a047fe7.el9.x86_64 130/156 2024-05-03T18:35:24.184 INFO:teuthology.orchestra.run.smithi113.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-03T18:35:24.184 INFO:teuthology.orchestra.run.smithi113.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-03T18:35:24.184 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-03T18:35:24.184 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-03T18:35:24.184 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-05-03T18:35:24.229 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-mgr-rook-2:19.0.0-3450.g9a047fe7.el9.noarch 131/156 2024-05-03T18:35:24.240 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3450.g9a047fe7.el9.noarch 131/156 2024-05-03T18:35:24.252 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-05-03T18:35:24.337 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-2:19.0.0-3450.g9a047fe7.el9.x86_64 132/156 2024-05-03T18:35:24.385 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-fuse-2:19.0.0-3450.g9a047fe7.el9.x86_64 133/156 2024-05-03T18:35:24.404 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-05-03T18:35:24.447 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-05-03T18:35:24.479 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-05-03T18:35:24.506 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-05-03T18:35:24.960 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: cephadm-2:19.0.0-3450.g9a047fe7.el9.noarch 123/156 2024-05-03T18:35:24.988 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : cephadm-2:19.0.0-3450.g9a047fe7.el9.noarch 123/156 2024-05-03T18:35:25.021 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3450.g9a047fe7.e 124/156 2024-05-03T18:35:25.093 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3450.g9a047fe7. 125/156 2024-05-03T18:35:25.120 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3450.g9a047fe7.el9.noa 126/156 2024-05-03T18:35:25.308 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-05-03T18:35:25.339 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-radosgw-2:19.0.0-3450.g9a047fe7.el9.x86_64 135/156 2024-05-03T18:35:25.361 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3450.g9a047fe7.el9.x86_64 135/156 2024-05-03T18:35:25.361 INFO:teuthology.orchestra.run.smithi113.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-03T18:35:25.361 INFO:teuthology.orchestra.run.smithi113.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-03T18:35:25.362 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-03T18:35:25.362 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-03T18:35:25.362 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-05-03T18:35:25.407 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3450.g9a047 136/156 2024-05-03T18:35:25.428 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3450.g9a047 136/156 2024-05-03T18:35:25.428 INFO:teuthology.orchestra.run.smithi113.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-03T18:35:25.428 INFO:teuthology.orchestra.run.smithi113.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-03T18:35:25.429 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-05-03T18:35:25.474 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : cephfs-mirror-2:19.0.0-3450.g9a047fe7.el9.x86_64 137/156 2024-05-03T18:35:25.496 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3450.g9a047fe7.el9.x86_64 137/156 2024-05-03T18:35:25.496 INFO:teuthology.orchestra.run.smithi113.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-03T18:35:25.496 INFO:teuthology.orchestra.run.smithi113.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-03T18:35:25.496 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-03T18:35:25.496 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-03T18:35:25.496 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-05-03T18:35:25.800 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : rbd-mirror-2:19.0.0-3450.g9a047fe7.el9.x86_64 138/156 2024-05-03T18:35:25.822 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3450.g9a047fe7.el9.x86_64 138/156 2024-05-03T18:35:25.823 INFO:teuthology.orchestra.run.smithi113.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-03T18:35:25.823 INFO:teuthology.orchestra.run.smithi113.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-03T18:35:25.823 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-03T18:35:25.823 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-03T18:35:25.823 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-05-03T18:35:26.224 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3450.g9a047fe7.el9.noa 126/156 2024-05-03T18:35:26.254 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3450.g9a047fe7.el9.n 127/156 2024-05-03T18:35:27.274 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3450.g9a047fe7.el9.n 127/156 2024-05-03T18:35:27.297 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3450.g9a0 128/156 2024-05-03T18:35:27.414 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3450.g9a0 128/156 2024-05-03T18:35:27.586 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3450.g9a047fe7.el 129/156 2024-05-03T18:35:27.607 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mgr-2:19.0.0-3450.g9a047fe7.el9.x86_64 130/156 2024-05-03T18:35:27.631 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3450.g9a047fe7.el9.x86_64 130/156 2024-05-03T18:35:27.632 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-03T18:35:27.632 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-03T18:35:27.632 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-03T18:35:27.632 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-03T18:35:27.632 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-03T18:35:27.683 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mgr-rook-2:19.0.0-3450.g9a047fe7.el9.noarch 131/156 2024-05-03T18:35:27.694 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3450.g9a047fe7.el9.noarch 131/156 2024-05-03T18:35:27.813 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-2:19.0.0-3450.g9a047fe7.el9.x86_64 132/156 2024-05-03T18:35:27.866 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-fuse-2:19.0.0-3450.g9a047fe7.el9.x86_64 133/156 2024-05-03T18:35:28.243 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : ceph-test-2:19.0.0-3450.g9a047fe7.el9.x86_64 139/156 2024-05-03T18:35:28.284 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : rbd-fuse-2:19.0.0-3450.g9a047fe7.el9.x86_64 140/156 2024-05-03T18:35:28.325 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : rbd-nbd-2:19.0.0-3450.g9a047fe7.el9.x86_64 141/156 2024-05-03T18:35:28.357 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-05-03T18:35:28.384 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-05-03T18:35:28.576 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : cephfs-shell-2:19.0.0-3450.g9a047fe7.el9.x86_64 144/156 2024-05-03T18:35:28.614 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-05-03T18:35:28.643 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : libcephfs-devel-2:19.0.0-3450.g9a047fe7.el9.x86_ 146/156 2024-05-03T18:35:28.843 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-05-03T18:35:28.869 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-radosgw-2:19.0.0-3450.g9a047fe7.el9.x86_64 135/156 2024-05-03T18:35:28.893 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3450.g9a047fe7.el9.x86_64 135/156 2024-05-03T18:35:28.893 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-03T18:35:28.893 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-03T18:35:28.893 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-03T18:35:28.893 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-03T18:35:28.893 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-03T18:35:28.936 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3450.g9a047 136/156 2024-05-03T18:35:28.959 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3450.g9a047 136/156 2024-05-03T18:35:28.959 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-03T18:35:28.960 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-03T18:35:28.960 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-03T18:35:29.012 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : cephfs-mirror-2:19.0.0-3450.g9a047fe7.el9.x86_64 137/156 2024-05-03T18:35:29.035 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3450.g9a047fe7.el9.x86_64 137/156 2024-05-03T18:35:29.036 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-03T18:35:29.036 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-03T18:35:29.036 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-03T18:35:29.036 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-03T18:35:29.036 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-03T18:35:29.267 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-05-03T18:35:29.307 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-05-03T18:35:29.328 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-05-03T18:35:29.362 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : rbd-mirror-2:19.0.0-3450.g9a047fe7.el9.x86_64 138/156 2024-05-03T18:35:29.386 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3450.g9a047fe7.el9.x86_64 138/156 2024-05-03T18:35:29.386 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-03T18:35:29.386 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-03T18:35:29.386 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-03T18:35:29.386 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-03T18:35:29.386 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-03T18:35:30.219 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : cephfs-top-2:19.0.0-3450.g9a047fe7.el9.noarch 148/156 2024-05-03T18:35:30.312 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-05-03T18:35:30.346 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-05-03T18:35:30.403 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : ceph-test-2:19.0.0-3450.g9a047fe7.el9.x86_64 139/156 2024-05-03T18:35:30.438 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-05-03T18:35:30.444 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : rbd-fuse-2:19.0.0-3450.g9a047fe7.el9.x86_64 140/156 2024-05-03T18:35:30.475 INFO:teuthology.orchestra.run.smithi032.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-05-03T18:35:30.475 INFO:teuthology.orchestra.run.smithi032.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-05-03T18:35:30.494 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : rbd-nbd-2:19.0.0-3450.g9a047fe7.el9.x86_64 141/156 2024-05-03T18:35:30.526 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-05-03T18:35:30.553 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-05-03T18:35:30.559 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-05-03T18:35:30.592 INFO:teuthology.orchestra.run.smithi032.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-05-03T18:35:30.592 INFO:teuthology.orchestra.run.smithi032.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-05-03T18:35:30.712 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-05-03T18:35:30.736 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : cephfs-shell-2:19.0.0-3450.g9a047fe7.el9.x86_64 144/156 2024-05-03T18:35:30.775 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-05-03T18:35:30.785 INFO:teuthology.orchestra.run.smithi032.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-05-03T18:35:30.805 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : libcephfs-devel-2:19.0.0-3450.g9a047fe7.el9.x86_ 146/156 2024-05-03T18:35:31.410 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-05-03T18:35:31.445 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-05-03T18:35:31.464 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-05-03T18:35:32.293 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : cephfs-top-2:19.0.0-3450.g9a047fe7.el9.noarch 148/156 2024-05-03T18:35:32.392 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-05-03T18:35:32.433 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-05-03T18:35:32.532 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-05-03T18:35:32.564 INFO:teuthology.orchestra.run.smithi113.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-05-03T18:35:32.564 INFO:teuthology.orchestra.run.smithi113.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-05-03T18:35:32.656 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-05-03T18:35:32.680 INFO:teuthology.orchestra.run.smithi113.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-05-03T18:35:32.681 INFO:teuthology.orchestra.run.smithi113.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-05-03T18:35:32.774 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-05-03T18:35:32.836 INFO:teuthology.orchestra.run.smithi113.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-05-03T18:35:34.283 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-test-2:19.0.0-3450.g9a047fe7.el9.x86_64 139/156 2024-05-03T18:35:34.325 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : rbd-fuse-2:19.0.0-3450.g9a047fe7.el9.x86_64 140/156 2024-05-03T18:35:34.366 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : rbd-nbd-2:19.0.0-3450.g9a047fe7.el9.x86_64 141/156 2024-05-03T18:35:34.397 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-05-03T18:35:34.424 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-05-03T18:35:34.625 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : cephfs-shell-2:19.0.0-3450.g9a047fe7.el9.x86_64 144/156 2024-05-03T18:35:34.670 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-05-03T18:35:34.700 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libcephfs-devel-2:19.0.0-3450.g9a047fe7.el9.x86_ 146/156 2024-05-03T18:35:35.318 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-05-03T18:35:35.356 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-05-03T18:35:35.377 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-05-03T18:35:36.298 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : cephfs-top-2:19.0.0-3450.g9a047fe7.el9.noarch 148/156 2024-05-03T18:35:36.394 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-05-03T18:35:36.438 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-05-03T18:35:36.472 INFO:teuthology.orchestra.run.smithi032.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-05-03T18:35:36.472 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-2:19.0.0-3450.g9a047fe7.el9.x86_64 1/156 2024-05-03T18:35:36.472 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-base-2:19.0.0-3450.g9a047fe7.el9.x86_64 2/156 2024-05-03T18:35:36.472 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-common-2:19.0.0-3450.g9a047fe7.el9.x86_64 3/156 2024-05-03T18:35:36.472 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-fuse-2:19.0.0-3450.g9a047fe7.el9.x86_64 4/156 2024-05-03T18:35:36.472 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3450.g9a047 5/156 2024-05-03T18:35:36.472 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-mds-2:19.0.0-3450.g9a047fe7.el9.x86_64 6/156 2024-05-03T18:35:36.472 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-mgr-2:19.0.0-3450.g9a047fe7.el9.x86_64 7/156 2024-05-03T18:35:36.472 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-mon-2:19.0.0-3450.g9a047fe7.el9.x86_64 8/156 2024-05-03T18:35:36.472 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-osd-2:19.0.0-3450.g9a047fe7.el9.x86_64 9/156 2024-05-03T18:35:36.473 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-radosgw-2:19.0.0-3450.g9a047fe7.el9.x86_64 10/156 2024-05-03T18:35:36.473 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-selinux-2:19.0.0-3450.g9a047fe7.el9.x86_64 11/156 2024-05-03T18:35:36.473 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-test-2:19.0.0-3450.g9a047fe7.el9.x86_64 12/156 2024-05-03T18:35:36.473 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : cephfs-mirror-2:19.0.0-3450.g9a047fe7.el9.x86_64 13/156 2024-05-03T18:35:36.473 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : cephfs-shell-2:19.0.0-3450.g9a047fe7.el9.x86_64 14/156 2024-05-03T18:35:36.473 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libcephfs-devel-2:19.0.0-3450.g9a047fe7.el9.x86_ 15/156 2024-05-03T18:35:36.473 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libcephfs2-2:19.0.0-3450.g9a047fe7.el9.x86_64 16/156 2024-05-03T18:35:36.473 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libcephsqlite-2:19.0.0-3450.g9a047fe7.el9.x86_64 17/156 2024-05-03T18:35:36.473 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : librados-devel-2:19.0.0-3450.g9a047fe7.el9.x86_6 18/156 2024-05-03T18:35:36.473 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libradosstriper1-2:19.0.0-3450.g9a047fe7.el9.x86 19/156 2024-05-03T18:35:36.473 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : librgw2-2:19.0.0-3450.g9a047fe7.el9.x86_64 20/156 2024-05-03T18:35:36.473 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3450.g9a047fe7.el 21/156 2024-05-03T18:35:36.473 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-ceph-common-2:19.0.0-3450.g9a047fe7.el9. 22/156 2024-05-03T18:35:36.473 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-cephfs-2:19.0.0-3450.g9a047fe7.el9.x86_6 23/156 2024-05-03T18:35:36.474 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-rados-2:19.0.0-3450.g9a047fe7.el9.x86_64 24/156 2024-05-03T18:35:36.474 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-rbd-2:19.0.0-3450.g9a047fe7.el9.x86_64 25/156 2024-05-03T18:35:36.474 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-rgw-2:19.0.0-3450.g9a047fe7.el9.x86_64 26/156 2024-05-03T18:35:36.474 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : rbd-fuse-2:19.0.0-3450.g9a047fe7.el9.x86_64 27/156 2024-05-03T18:35:36.486 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : rbd-mirror-2:19.0.0-3450.g9a047fe7.el9.x86_64 28/156 2024-05-03T18:35:36.486 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : rbd-nbd-2:19.0.0-3450.g9a047fe7.el9.x86_64 29/156 2024-05-03T18:35:36.486 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3450.g9a047fe7. 30/156 2024-05-03T18:35:36.486 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3450.g9a047fe7.el9.noa 31/156 2024-05-03T18:35:36.486 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3450.g9a047fe7.el9.n 32/156 2024-05-03T18:35:36.487 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3450.g9a0 33/156 2024-05-03T18:35:36.487 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3450.g9a047fe7.el 34/156 2024-05-03T18:35:36.487 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3450.g9a047fe7.el9.noarch 35/156 2024-05-03T18:35:36.487 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3450.g9a047fe7.e 36/156 2024-05-03T18:35:36.487 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : ceph-volume-2:19.0.0-3450.g9a047fe7.el9.noarch 37/156 2024-05-03T18:35:36.487 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : cephadm-2:19.0.0-3450.g9a047fe7.el9.noarch 38/156 2024-05-03T18:35:36.487 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : cephfs-top-2:19.0.0-3450.g9a047fe7.el9.noarch 39/156 2024-05-03T18:35:36.487 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-05-03T18:35:36.487 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-05-03T18:35:36.487 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-05-03T18:35:36.487 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-05-03T18:35:36.487 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-05-03T18:35:36.487 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-05-03T18:35:36.488 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-05-03T18:35:36.488 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-05-03T18:35:36.488 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-05-03T18:35:36.488 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-05-03T18:35:36.488 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-05-03T18:35:36.488 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-05-03T18:35:36.488 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-05-03T18:35:36.488 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-05-03T18:35:36.488 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-05-03T18:35:36.488 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-05-03T18:35:36.488 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-05-03T18:35:36.488 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-05-03T18:35:36.489 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-05-03T18:35:36.489 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-05-03T18:35:36.489 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-05-03T18:35:36.489 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-05-03T18:35:36.489 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-05-03T18:35:36.489 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-05-03T18:35:36.489 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-05-03T18:35:36.489 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-05-03T18:35:36.489 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-05-03T18:35:36.489 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-05-03T18:35:36.489 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-05-03T18:35:36.489 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-05-03T18:35:36.489 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-05-03T18:35:36.490 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-05-03T18:35:36.490 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-05-03T18:35:36.490 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-05-03T18:35:36.490 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-05-03T18:35:36.490 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-05-03T18:35:36.490 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-05-03T18:35:36.490 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-05-03T18:35:36.490 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-05-03T18:35:36.490 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-05-03T18:35:36.490 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-05-03T18:35:36.490 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-05-03T18:35:36.490 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-05-03T18:35:36.490 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-05-03T18:35:36.491 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-05-03T18:35:36.491 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-05-03T18:35:36.491 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-05-03T18:35:36.491 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-05-03T18:35:36.491 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-05-03T18:35:36.491 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-05-03T18:35:36.491 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-05-03T18:35:36.491 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-05-03T18:35:36.491 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-05-03T18:35:36.491 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-05-03T18:35:36.491 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-05-03T18:35:36.491 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-05-03T18:35:36.491 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-05-03T18:35:36.491 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-05-03T18:35:36.492 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-05-03T18:35:36.492 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-05-03T18:35:36.492 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-05-03T18:35:36.492 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-05-03T18:35:36.492 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-05-03T18:35:36.492 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-05-03T18:35:36.492 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-05-03T18:35:36.492 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-05-03T18:35:36.492 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-05-03T18:35:36.492 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-05-03T18:35:36.492 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/156 2024-05-03T18:35:36.492 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/156 2024-05-03T18:35:36.492 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-05-03T18:35:36.492 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-05-03T18:35:36.493 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-05-03T18:35:36.493 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/156 2024-05-03T18:35:36.493 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-05-03T18:35:36.493 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-05-03T18:35:36.493 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-05-03T18:35:36.493 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-05-03T18:35:36.493 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-05-03T18:35:36.493 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-05-03T18:35:36.493 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-05-03T18:35:36.493 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-05-03T18:35:36.493 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-05-03T18:35:36.493 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-05-03T18:35:36.493 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-05-03T18:35:36.493 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-05-03T18:35:36.494 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-05-03T18:35:36.494 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-05-03T18:35:36.494 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-05-03T18:35:36.494 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-05-03T18:35:36.494 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-05-03T18:35:36.494 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-05-03T18:35:36.494 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-05-03T18:35:36.494 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-05-03T18:35:36.494 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-05-03T18:35:36.494 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-05-03T18:35:36.494 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-05-03T18:35:36.494 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-05-03T18:35:36.495 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-05-03T18:35:36.495 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-05-03T18:35:36.495 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-05-03T18:35:36.495 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-05-03T18:35:36.495 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-05-03T18:35:36.495 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-05-03T18:35:36.495 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-05-03T18:35:36.495 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-05-03T18:35:36.495 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-05-03T18:35:36.495 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-05-03T18:35:36.495 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-05-03T18:35:36.495 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : librados2-2:19.0.0-3450.g9a047fe7.el9.x86_64 149/156 2024-05-03T18:35:36.495 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-05-03T18:35:36.496 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : librbd1-2:19.0.0-3450.g9a047fe7.el9.x86_64 151/156 2024-05-03T18:35:36.496 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-05-03T18:35:36.496 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-05-03T18:35:36.496 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-05-03T18:35:36.496 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-05-03T18:35:36.531 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-05-03T18:35:36.565 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-05-03T18:35:36.565 INFO:teuthology.orchestra.run.smithi045.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-05-03T18:35:36.633 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-05-03T18:35:36.665 INFO:teuthology.orchestra.run.smithi045.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-05-03T18:35:36.666 INFO:teuthology.orchestra.run.smithi045.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-05-03T18:35:36.742 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-05-03T18:35:36.817 INFO:teuthology.orchestra.run.smithi045.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-05-03T18:35:38.470 INFO:teuthology.orchestra.run.smithi032.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-05-03T18:35:38.470 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-03T18:35:38.470 INFO:teuthology.orchestra.run.smithi032.stdout:Upgraded: 2024-05-03T18:35:38.470 INFO:teuthology.orchestra.run.smithi032.stdout: librados2-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:35:38.470 INFO:teuthology.orchestra.run.smithi032.stdout: librbd1-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:35:38.470 INFO:teuthology.orchestra.run.smithi032.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-03T18:35:38.470 INFO:teuthology.orchestra.run.smithi032.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-03T18:35:38.471 INFO:teuthology.orchestra.run.smithi032.stdout:Installed: 2024-05-03T18:35:38.471 INFO:teuthology.orchestra.run.smithi032.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-03T18:35:38.471 INFO:teuthology.orchestra.run.smithi032.stdout: automake-1.16.2-8.el9.noarch 2024-05-03T18:35:38.471 INFO:teuthology.orchestra.run.smithi032.stdout: bison-3.7.4-5.el9.x86_64 2024-05-03T18:35:38.471 INFO:teuthology.orchestra.run.smithi032.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-03T18:35:38.471 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:35:38.471 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-base-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:35:38.471 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-common-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:35:38.471 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-fuse-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:35:38.471 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-grafana-dashboards-2:19.0.0-3450.g9a047fe7.el9.noarch 2024-05-03T18:35:38.472 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-immutable-object-cache-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:35:38.472 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mds-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:35:38.472 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:35:38.472 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-cephadm-2:19.0.0-3450.g9a047fe7.el9.noarch 2024-05-03T18:35:38.472 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-dashboard-2:19.0.0-3450.g9a047fe7.el9.noarch 2024-05-03T18:35:38.472 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3450.g9a047fe7.el9.noarch 2024-05-03T18:35:38.472 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-modules-core-2:19.0.0-3450.g9a047fe7.el9.noarch 2024-05-03T18:35:38.472 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-rook-2:19.0.0-3450.g9a047fe7.el9.noarch 2024-05-03T18:35:38.472 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mon-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:35:38.472 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-osd-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:35:38.472 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-prometheus-alerts-2:19.0.0-3450.g9a047fe7.el9.noarch 2024-05-03T18:35:38.472 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-radosgw-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:35:38.472 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-selinux-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:35:38.472 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-test-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:35:38.473 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-volume-2:19.0.0-3450.g9a047fe7.el9.noarch 2024-05-03T18:35:38.473 INFO:teuthology.orchestra.run.smithi032.stdout: cephadm-2:19.0.0-3450.g9a047fe7.el9.noarch 2024-05-03T18:35:38.473 INFO:teuthology.orchestra.run.smithi032.stdout: cephfs-mirror-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:35:38.473 INFO:teuthology.orchestra.run.smithi032.stdout: cephfs-shell-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:35:38.473 INFO:teuthology.orchestra.run.smithi032.stdout: cephfs-top-2:19.0.0-3450.g9a047fe7.el9.noarch 2024-05-03T18:35:38.473 INFO:teuthology.orchestra.run.smithi032.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-05-03T18:35:38.473 INFO:teuthology.orchestra.run.smithi032.stdout: flex-2.6.4-9.el9.x86_64 2024-05-03T18:35:38.473 INFO:teuthology.orchestra.run.smithi032.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-03T18:35:38.473 INFO:teuthology.orchestra.run.smithi032.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-03T18:35:38.473 INFO:teuthology.orchestra.run.smithi032.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-03T18:35:38.473 INFO:teuthology.orchestra.run.smithi032.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-03T18:35:38.473 INFO:teuthology.orchestra.run.smithi032.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-03T18:35:38.473 INFO:teuthology.orchestra.run.smithi032.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-03T18:35:38.474 INFO:teuthology.orchestra.run.smithi032.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-03T18:35:38.474 INFO:teuthology.orchestra.run.smithi032.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-03T18:35:38.474 INFO:teuthology.orchestra.run.smithi032.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-03T18:35:38.474 INFO:teuthology.orchestra.run.smithi032.stdout: libcephfs-devel-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:35:38.474 INFO:teuthology.orchestra.run.smithi032.stdout: libcephfs2-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:35:38.474 INFO:teuthology.orchestra.run.smithi032.stdout: libcephsqlite-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:35:38.474 INFO:teuthology.orchestra.run.smithi032.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-03T18:35:38.474 INFO:teuthology.orchestra.run.smithi032.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-03T18:35:38.474 INFO:teuthology.orchestra.run.smithi032.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-03T18:35:38.474 INFO:teuthology.orchestra.run.smithi032.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-03T18:35:38.474 INFO:teuthology.orchestra.run.smithi032.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-03T18:35:38.474 INFO:teuthology.orchestra.run.smithi032.stdout: librados-devel-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:35:38.474 INFO:teuthology.orchestra.run.smithi032.stdout: libradosstriper1-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:35:38.474 INFO:teuthology.orchestra.run.smithi032.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-03T18:35:38.474 INFO:teuthology.orchestra.run.smithi032.stdout: librgw2-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:35:38.475 INFO:teuthology.orchestra.run.smithi032.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-03T18:35:38.475 INFO:teuthology.orchestra.run.smithi032.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-03T18:35:38.475 INFO:teuthology.orchestra.run.smithi032.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-03T18:35:38.475 INFO:teuthology.orchestra.run.smithi032.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-03T18:35:38.475 INFO:teuthology.orchestra.run.smithi032.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-03T18:35:38.475 INFO:teuthology.orchestra.run.smithi032.stdout: lua-5.4.4-4.el9.x86_64 2024-05-03T18:35:38.475 INFO:teuthology.orchestra.run.smithi032.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-03T18:35:38.475 INFO:teuthology.orchestra.run.smithi032.stdout: luarocks-3.9.2-1.el9.noarch 2024-05-03T18:35:38.475 INFO:teuthology.orchestra.run.smithi032.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-03T18:35:38.475 INFO:teuthology.orchestra.run.smithi032.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-03T18:35:38.475 INFO:teuthology.orchestra.run.smithi032.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-03T18:35:38.475 INFO:teuthology.orchestra.run.smithi032.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-03T18:35:38.475 INFO:teuthology.orchestra.run.smithi032.stdout: postgresql-13.14-1.el9.x86_64 2024-05-03T18:35:38.477 INFO:teuthology.orchestra.run.smithi032.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-03T18:35:38.477 INFO:teuthology.orchestra.run.smithi032.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-03T18:35:38.478 INFO:teuthology.orchestra.run.smithi032.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-03T18:35:38.478 INFO:teuthology.orchestra.run.smithi032.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-03T18:35:38.478 INFO:teuthology.orchestra.run.smithi032.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-03T18:35:38.478 INFO:teuthology.orchestra.run.smithi032.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-03T18:35:38.478 INFO:teuthology.orchestra.run.smithi032.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-03T18:35:38.478 INFO:teuthology.orchestra.run.smithi032.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-03T18:35:38.478 INFO:teuthology.orchestra.run.smithi032.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-03T18:35:38.478 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-03T18:35:38.478 INFO:teuthology.orchestra.run.smithi032.stdout: python3-ceph-argparse-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:35:38.478 INFO:teuthology.orchestra.run.smithi032.stdout: python3-ceph-common-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:35:38.478 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cephfs-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:35:38.478 INFO:teuthology.orchestra.run.smithi032.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-03T18:35:38.479 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-03T18:35:38.479 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-03T18:35:38.479 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-03T18:35:38.479 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-03T18:35:38.479 INFO:teuthology.orchestra.run.smithi032.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-05-03T18:35:38.479 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-03T18:35:38.479 INFO:teuthology.orchestra.run.smithi032.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-03T18:35:38.479 INFO:teuthology.orchestra.run.smithi032.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-03T18:35:38.479 INFO:teuthology.orchestra.run.smithi032.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-03T18:35:38.479 INFO:teuthology.orchestra.run.smithi032.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-03T18:35:38.479 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-03T18:35:38.479 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-03T18:35:38.479 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-03T18:35:38.479 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-03T18:35:38.479 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-03T18:35:38.479 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-03T18:35:38.480 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-03T18:35:38.480 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-03T18:35:38.480 INFO:teuthology.orchestra.run.smithi032.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-03T18:35:38.480 INFO:teuthology.orchestra.run.smithi032.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-03T18:35:38.480 INFO:teuthology.orchestra.run.smithi032.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-03T18:35:38.480 INFO:teuthology.orchestra.run.smithi032.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-03T18:35:38.480 INFO:teuthology.orchestra.run.smithi032.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-03T18:35:38.480 INFO:teuthology.orchestra.run.smithi032.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-03T18:35:38.480 INFO:teuthology.orchestra.run.smithi032.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-03T18:35:38.480 INFO:teuthology.orchestra.run.smithi032.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-03T18:35:38.480 INFO:teuthology.orchestra.run.smithi032.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-03T18:35:38.480 INFO:teuthology.orchestra.run.smithi032.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-03T18:35:38.480 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-03T18:35:38.480 INFO:teuthology.orchestra.run.smithi032.stdout: python3-ply-3.11-14.el9.noarch 2024-05-03T18:35:38.481 INFO:teuthology.orchestra.run.smithi032.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-03T18:35:38.481 INFO:teuthology.orchestra.run.smithi032.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-03T18:35:38.481 INFO:teuthology.orchestra.run.smithi032.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-03T18:35:38.481 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-03T18:35:38.481 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-03T18:35:38.481 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-03T18:35:38.481 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-03T18:35:38.481 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-03T18:35:38.481 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-03T18:35:38.481 INFO:teuthology.orchestra.run.smithi032.stdout: python3-rados-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:35:38.481 INFO:teuthology.orchestra.run.smithi032.stdout: python3-rbd-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:35:38.481 INFO:teuthology.orchestra.run.smithi032.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-03T18:35:38.482 INFO:teuthology.orchestra.run.smithi032.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-03T18:35:38.482 INFO:teuthology.orchestra.run.smithi032.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-03T18:35:38.482 INFO:teuthology.orchestra.run.smithi032.stdout: python3-rgw-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:35:38.483 INFO:teuthology.orchestra.run.smithi032.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-03T18:35:38.483 INFO:teuthology.orchestra.run.smithi032.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-03T18:35:38.483 INFO:teuthology.orchestra.run.smithi032.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-03T18:35:38.483 INFO:teuthology.orchestra.run.smithi032.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-03T18:35:38.483 INFO:teuthology.orchestra.run.smithi032.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-03T18:35:38.483 INFO:teuthology.orchestra.run.smithi032.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-03T18:35:38.483 INFO:teuthology.orchestra.run.smithi032.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-03T18:35:38.483 INFO:teuthology.orchestra.run.smithi032.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-03T18:35:38.483 INFO:teuthology.orchestra.run.smithi032.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-03T18:35:38.483 INFO:teuthology.orchestra.run.smithi032.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-03T18:35:38.483 INFO:teuthology.orchestra.run.smithi032.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-03T18:35:38.483 INFO:teuthology.orchestra.run.smithi032.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-03T18:35:38.483 INFO:teuthology.orchestra.run.smithi032.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-03T18:35:38.484 INFO:teuthology.orchestra.run.smithi032.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-03T18:35:38.484 INFO:teuthology.orchestra.run.smithi032.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-03T18:35:38.484 INFO:teuthology.orchestra.run.smithi032.stdout: rbd-fuse-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:35:38.484 INFO:teuthology.orchestra.run.smithi032.stdout: rbd-mirror-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:35:38.484 INFO:teuthology.orchestra.run.smithi032.stdout: rbd-nbd-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:35:38.484 INFO:teuthology.orchestra.run.smithi032.stdout: re2-1:20211101-3.el9.x86_64 2024-05-03T18:35:38.484 INFO:teuthology.orchestra.run.smithi032.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-03T18:35:38.484 INFO:teuthology.orchestra.run.smithi032.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-03T18:35:38.484 INFO:teuthology.orchestra.run.smithi032.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-03T18:35:38.484 INFO:teuthology.orchestra.run.smithi032.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-03T18:35:38.484 INFO:teuthology.orchestra.run.smithi032.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-03T18:35:38.485 INFO:teuthology.orchestra.run.smithi032.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-03T18:35:38.485 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-03T18:35:38.485 INFO:teuthology.orchestra.run.smithi032.stdout:Complete! 2024-05-03T18:35:38.726 INFO:teuthology.orchestra.run.smithi113.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-05-03T18:35:38.726 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-2:19.0.0-3450.g9a047fe7.el9.x86_64 1/156 2024-05-03T18:35:38.726 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-base-2:19.0.0-3450.g9a047fe7.el9.x86_64 2/156 2024-05-03T18:35:38.726 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-common-2:19.0.0-3450.g9a047fe7.el9.x86_64 3/156 2024-05-03T18:35:38.726 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-fuse-2:19.0.0-3450.g9a047fe7.el9.x86_64 4/156 2024-05-03T18:35:38.726 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3450.g9a047 5/156 2024-05-03T18:35:38.726 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-mds-2:19.0.0-3450.g9a047fe7.el9.x86_64 6/156 2024-05-03T18:35:38.726 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-mgr-2:19.0.0-3450.g9a047fe7.el9.x86_64 7/156 2024-05-03T18:35:38.726 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-mon-2:19.0.0-3450.g9a047fe7.el9.x86_64 8/156 2024-05-03T18:35:38.727 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-osd-2:19.0.0-3450.g9a047fe7.el9.x86_64 9/156 2024-05-03T18:35:38.727 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-radosgw-2:19.0.0-3450.g9a047fe7.el9.x86_64 10/156 2024-05-03T18:35:38.727 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-selinux-2:19.0.0-3450.g9a047fe7.el9.x86_64 11/156 2024-05-03T18:35:38.727 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-test-2:19.0.0-3450.g9a047fe7.el9.x86_64 12/156 2024-05-03T18:35:38.727 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : cephfs-mirror-2:19.0.0-3450.g9a047fe7.el9.x86_64 13/156 2024-05-03T18:35:38.727 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : cephfs-shell-2:19.0.0-3450.g9a047fe7.el9.x86_64 14/156 2024-05-03T18:35:38.727 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libcephfs-devel-2:19.0.0-3450.g9a047fe7.el9.x86_ 15/156 2024-05-03T18:35:38.727 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libcephfs2-2:19.0.0-3450.g9a047fe7.el9.x86_64 16/156 2024-05-03T18:35:38.727 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libcephsqlite-2:19.0.0-3450.g9a047fe7.el9.x86_64 17/156 2024-05-03T18:35:38.727 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : librados-devel-2:19.0.0-3450.g9a047fe7.el9.x86_6 18/156 2024-05-03T18:35:38.727 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libradosstriper1-2:19.0.0-3450.g9a047fe7.el9.x86 19/156 2024-05-03T18:35:38.727 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : librgw2-2:19.0.0-3450.g9a047fe7.el9.x86_64 20/156 2024-05-03T18:35:38.727 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3450.g9a047fe7.el 21/156 2024-05-03T18:35:38.727 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-ceph-common-2:19.0.0-3450.g9a047fe7.el9. 22/156 2024-05-03T18:35:38.727 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-cephfs-2:19.0.0-3450.g9a047fe7.el9.x86_6 23/156 2024-05-03T18:35:38.728 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-rados-2:19.0.0-3450.g9a047fe7.el9.x86_64 24/156 2024-05-03T18:35:38.728 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-rbd-2:19.0.0-3450.g9a047fe7.el9.x86_64 25/156 2024-05-03T18:35:38.728 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-rgw-2:19.0.0-3450.g9a047fe7.el9.x86_64 26/156 2024-05-03T18:35:38.728 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : rbd-fuse-2:19.0.0-3450.g9a047fe7.el9.x86_64 27/156 2024-05-03T18:35:38.728 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : rbd-mirror-2:19.0.0-3450.g9a047fe7.el9.x86_64 28/156 2024-05-03T18:35:38.728 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : rbd-nbd-2:19.0.0-3450.g9a047fe7.el9.x86_64 29/156 2024-05-03T18:35:38.728 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3450.g9a047fe7. 30/156 2024-05-03T18:35:38.740 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3450.g9a047fe7.el9.noa 31/156 2024-05-03T18:35:38.740 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3450.g9a047fe7.el9.n 32/156 2024-05-03T18:35:38.740 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3450.g9a0 33/156 2024-05-03T18:35:38.740 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3450.g9a047fe7.el 34/156 2024-05-03T18:35:38.740 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3450.g9a047fe7.el9.noarch 35/156 2024-05-03T18:35:38.740 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3450.g9a047fe7.e 36/156 2024-05-03T18:35:38.740 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : ceph-volume-2:19.0.0-3450.g9a047fe7.el9.noarch 37/156 2024-05-03T18:35:38.740 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : cephadm-2:19.0.0-3450.g9a047fe7.el9.noarch 38/156 2024-05-03T18:35:38.740 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : cephfs-top-2:19.0.0-3450.g9a047fe7.el9.noarch 39/156 2024-05-03T18:35:38.740 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-05-03T18:35:38.740 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-05-03T18:35:38.740 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-05-03T18:35:38.740 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-05-03T18:35:38.741 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-05-03T18:35:38.741 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-05-03T18:35:38.741 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-05-03T18:35:38.741 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-05-03T18:35:38.741 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-05-03T18:35:38.741 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-05-03T18:35:38.741 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-05-03T18:35:38.741 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-05-03T18:35:38.741 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-05-03T18:35:38.741 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-05-03T18:35:38.741 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-05-03T18:35:38.741 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-05-03T18:35:38.741 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-05-03T18:35:38.741 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-05-03T18:35:38.741 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-05-03T18:35:38.742 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-05-03T18:35:38.742 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-05-03T18:35:38.742 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-05-03T18:35:38.742 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-05-03T18:35:38.742 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-05-03T18:35:38.742 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-05-03T18:35:38.742 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-05-03T18:35:38.742 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-05-03T18:35:38.742 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-05-03T18:35:38.742 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-05-03T18:35:38.742 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-05-03T18:35:38.742 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-05-03T18:35:38.742 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-05-03T18:35:38.742 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-05-03T18:35:38.743 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-05-03T18:35:38.743 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-05-03T18:35:38.743 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-05-03T18:35:38.743 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-05-03T18:35:38.743 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-05-03T18:35:38.743 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-05-03T18:35:38.743 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-05-03T18:35:38.743 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-05-03T18:35:38.743 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-05-03T18:35:38.743 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-05-03T18:35:38.743 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-05-03T18:35:38.743 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-05-03T18:35:38.743 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-05-03T18:35:38.743 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-05-03T18:35:38.744 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-05-03T18:35:38.744 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-05-03T18:35:38.744 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-05-03T18:35:38.744 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-05-03T18:35:38.744 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-05-03T18:35:38.744 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-05-03T18:35:38.744 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-05-03T18:35:38.744 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-05-03T18:35:38.744 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-05-03T18:35:38.744 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-05-03T18:35:38.744 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-05-03T18:35:38.744 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-05-03T18:35:38.744 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-05-03T18:35:38.744 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-05-03T18:35:38.744 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-05-03T18:35:38.745 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-05-03T18:35:38.745 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-05-03T18:35:38.745 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-05-03T18:35:38.745 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-05-03T18:35:38.745 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-05-03T18:35:38.745 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-05-03T18:35:38.745 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/156 2024-05-03T18:35:38.745 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/156 2024-05-03T18:35:38.745 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-05-03T18:35:38.745 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-05-03T18:35:38.745 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-05-03T18:35:38.745 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/156 2024-05-03T18:35:38.745 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-05-03T18:35:38.745 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-05-03T18:35:38.745 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-05-03T18:35:38.746 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-05-03T18:35:38.746 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-05-03T18:35:38.746 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-05-03T18:35:38.746 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-05-03T18:35:38.746 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-05-03T18:35:38.746 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-05-03T18:35:38.746 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-05-03T18:35:38.746 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-05-03T18:35:38.746 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-05-03T18:35:38.746 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-05-03T18:35:38.746 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-05-03T18:35:38.746 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-05-03T18:35:38.746 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-05-03T18:35:38.746 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-05-03T18:35:38.747 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-05-03T18:35:38.747 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-05-03T18:35:38.747 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-05-03T18:35:38.747 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-05-03T18:35:38.747 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-05-03T18:35:38.747 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-05-03T18:35:38.747 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-05-03T18:35:38.747 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-05-03T18:35:38.747 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-05-03T18:35:38.747 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-05-03T18:35:38.747 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-05-03T18:35:38.747 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-05-03T18:35:38.747 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-05-03T18:35:38.747 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-05-03T18:35:38.748 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-05-03T18:35:38.748 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-05-03T18:35:38.748 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-05-03T18:35:38.748 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-05-03T18:35:38.748 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : librados2-2:19.0.0-3450.g9a047fe7.el9.x86_64 149/156 2024-05-03T18:35:38.748 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-05-03T18:35:38.748 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : librbd1-2:19.0.0-3450.g9a047fe7.el9.x86_64 151/156 2024-05-03T18:35:38.748 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-05-03T18:35:38.748 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-05-03T18:35:38.748 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-05-03T18:35:38.748 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-05-03T18:35:38.881 DEBUG:teuthology.parallel:result is None 2024-05-03T18:35:40.861 INFO:teuthology.orchestra.run.smithi113.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-05-03T18:35:40.862 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-05-03T18:35:40.862 INFO:teuthology.orchestra.run.smithi113.stdout:Upgraded: 2024-05-03T18:35:40.862 INFO:teuthology.orchestra.run.smithi113.stdout: librados2-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:35:40.862 INFO:teuthology.orchestra.run.smithi113.stdout: librbd1-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:35:40.862 INFO:teuthology.orchestra.run.smithi113.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-03T18:35:40.862 INFO:teuthology.orchestra.run.smithi113.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-03T18:35:40.862 INFO:teuthology.orchestra.run.smithi113.stdout:Installed: 2024-05-03T18:35:40.862 INFO:teuthology.orchestra.run.smithi113.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-03T18:35:40.862 INFO:teuthology.orchestra.run.smithi113.stdout: automake-1.16.2-8.el9.noarch 2024-05-03T18:35:40.862 INFO:teuthology.orchestra.run.smithi113.stdout: bison-3.7.4-5.el9.x86_64 2024-05-03T18:35:40.862 INFO:teuthology.orchestra.run.smithi113.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-03T18:35:40.862 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:35:40.863 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-base-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:35:40.863 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-common-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:35:40.863 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-fuse-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:35:40.863 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-grafana-dashboards-2:19.0.0-3450.g9a047fe7.el9.noarch 2024-05-03T18:35:40.863 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-immutable-object-cache-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:35:40.863 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mds-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:35:40.863 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mgr-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:35:40.863 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mgr-cephadm-2:19.0.0-3450.g9a047fe7.el9.noarch 2024-05-03T18:35:40.863 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mgr-dashboard-2:19.0.0-3450.g9a047fe7.el9.noarch 2024-05-03T18:35:40.863 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3450.g9a047fe7.el9.noarch 2024-05-03T18:35:40.863 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mgr-modules-core-2:19.0.0-3450.g9a047fe7.el9.noarch 2024-05-03T18:35:40.863 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mgr-rook-2:19.0.0-3450.g9a047fe7.el9.noarch 2024-05-03T18:35:40.863 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mon-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:35:40.863 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-osd-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:35:40.864 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-prometheus-alerts-2:19.0.0-3450.g9a047fe7.el9.noarch 2024-05-03T18:35:40.864 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-radosgw-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:35:40.864 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-selinux-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:35:40.864 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-test-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:35:40.864 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-volume-2:19.0.0-3450.g9a047fe7.el9.noarch 2024-05-03T18:35:40.864 INFO:teuthology.orchestra.run.smithi113.stdout: cephadm-2:19.0.0-3450.g9a047fe7.el9.noarch 2024-05-03T18:35:40.864 INFO:teuthology.orchestra.run.smithi113.stdout: cephfs-mirror-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:35:40.864 INFO:teuthology.orchestra.run.smithi113.stdout: cephfs-shell-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:35:40.864 INFO:teuthology.orchestra.run.smithi113.stdout: cephfs-top-2:19.0.0-3450.g9a047fe7.el9.noarch 2024-05-03T18:35:40.864 INFO:teuthology.orchestra.run.smithi113.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-05-03T18:35:40.864 INFO:teuthology.orchestra.run.smithi113.stdout: flex-2.6.4-9.el9.x86_64 2024-05-03T18:35:40.864 INFO:teuthology.orchestra.run.smithi113.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-03T18:35:40.864 INFO:teuthology.orchestra.run.smithi113.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-03T18:35:40.865 INFO:teuthology.orchestra.run.smithi113.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-03T18:35:40.865 INFO:teuthology.orchestra.run.smithi113.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-03T18:35:40.865 INFO:teuthology.orchestra.run.smithi113.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-03T18:35:40.865 INFO:teuthology.orchestra.run.smithi113.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-03T18:35:40.865 INFO:teuthology.orchestra.run.smithi113.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-03T18:35:40.865 INFO:teuthology.orchestra.run.smithi113.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-03T18:35:40.865 INFO:teuthology.orchestra.run.smithi113.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-03T18:35:40.865 INFO:teuthology.orchestra.run.smithi113.stdout: libcephfs-devel-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:35:40.865 INFO:teuthology.orchestra.run.smithi113.stdout: libcephfs2-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:35:40.865 INFO:teuthology.orchestra.run.smithi113.stdout: libcephsqlite-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:35:40.865 INFO:teuthology.orchestra.run.smithi113.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-03T18:35:40.865 INFO:teuthology.orchestra.run.smithi113.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-03T18:35:40.865 INFO:teuthology.orchestra.run.smithi113.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-03T18:35:40.865 INFO:teuthology.orchestra.run.smithi113.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-03T18:35:40.866 INFO:teuthology.orchestra.run.smithi113.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-03T18:35:40.866 INFO:teuthology.orchestra.run.smithi113.stdout: librados-devel-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:35:40.866 INFO:teuthology.orchestra.run.smithi113.stdout: libradosstriper1-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:35:40.866 INFO:teuthology.orchestra.run.smithi113.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-03T18:35:40.866 INFO:teuthology.orchestra.run.smithi113.stdout: librgw2-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:35:40.866 INFO:teuthology.orchestra.run.smithi113.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-03T18:35:40.866 INFO:teuthology.orchestra.run.smithi113.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-03T18:35:40.866 INFO:teuthology.orchestra.run.smithi113.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-03T18:35:40.866 INFO:teuthology.orchestra.run.smithi113.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-03T18:35:40.866 INFO:teuthology.orchestra.run.smithi113.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-03T18:35:40.866 INFO:teuthology.orchestra.run.smithi113.stdout: lua-5.4.4-4.el9.x86_64 2024-05-03T18:35:40.866 INFO:teuthology.orchestra.run.smithi113.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-03T18:35:40.866 INFO:teuthology.orchestra.run.smithi113.stdout: luarocks-3.9.2-1.el9.noarch 2024-05-03T18:35:40.867 INFO:teuthology.orchestra.run.smithi113.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-03T18:35:40.867 INFO:teuthology.orchestra.run.smithi113.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-03T18:35:40.867 INFO:teuthology.orchestra.run.smithi113.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-03T18:35:40.867 INFO:teuthology.orchestra.run.smithi113.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-03T18:35:40.867 INFO:teuthology.orchestra.run.smithi113.stdout: postgresql-13.14-1.el9.x86_64 2024-05-03T18:35:40.867 INFO:teuthology.orchestra.run.smithi113.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-03T18:35:40.867 INFO:teuthology.orchestra.run.smithi113.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-03T18:35:40.867 INFO:teuthology.orchestra.run.smithi113.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-03T18:35:40.867 INFO:teuthology.orchestra.run.smithi113.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-03T18:35:40.867 INFO:teuthology.orchestra.run.smithi113.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-03T18:35:40.867 INFO:teuthology.orchestra.run.smithi113.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-03T18:35:40.867 INFO:teuthology.orchestra.run.smithi113.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-03T18:35:40.867 INFO:teuthology.orchestra.run.smithi113.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-03T18:35:40.867 INFO:teuthology.orchestra.run.smithi113.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-03T18:35:40.868 INFO:teuthology.orchestra.run.smithi113.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-03T18:35:40.868 INFO:teuthology.orchestra.run.smithi113.stdout: python3-ceph-argparse-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:35:40.868 INFO:teuthology.orchestra.run.smithi113.stdout: python3-ceph-common-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:35:40.868 INFO:teuthology.orchestra.run.smithi113.stdout: python3-cephfs-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:35:40.868 INFO:teuthology.orchestra.run.smithi113.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-03T18:35:40.868 INFO:teuthology.orchestra.run.smithi113.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-03T18:35:40.868 INFO:teuthology.orchestra.run.smithi113.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-03T18:35:40.868 INFO:teuthology.orchestra.run.smithi113.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-03T18:35:40.868 INFO:teuthology.orchestra.run.smithi113.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-03T18:35:40.868 INFO:teuthology.orchestra.run.smithi113.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-05-03T18:35:40.868 INFO:teuthology.orchestra.run.smithi113.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-03T18:35:40.868 INFO:teuthology.orchestra.run.smithi113.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-03T18:35:40.868 INFO:teuthology.orchestra.run.smithi113.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-03T18:35:40.869 INFO:teuthology.orchestra.run.smithi113.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-03T18:35:40.869 INFO:teuthology.orchestra.run.smithi113.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-03T18:35:40.869 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-03T18:35:40.869 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-03T18:35:40.869 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-03T18:35:40.869 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-03T18:35:40.869 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-03T18:35:40.869 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-03T18:35:40.869 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-03T18:35:40.869 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-03T18:35:40.869 INFO:teuthology.orchestra.run.smithi113.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-03T18:35:40.869 INFO:teuthology.orchestra.run.smithi113.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-03T18:35:40.869 INFO:teuthology.orchestra.run.smithi113.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-03T18:35:40.869 INFO:teuthology.orchestra.run.smithi113.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-03T18:35:40.870 INFO:teuthology.orchestra.run.smithi113.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-03T18:35:40.870 INFO:teuthology.orchestra.run.smithi113.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-03T18:35:40.870 INFO:teuthology.orchestra.run.smithi113.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-03T18:35:40.870 INFO:teuthology.orchestra.run.smithi113.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-03T18:35:40.870 INFO:teuthology.orchestra.run.smithi113.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-03T18:35:40.870 INFO:teuthology.orchestra.run.smithi113.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-03T18:35:40.870 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-03T18:35:40.870 INFO:teuthology.orchestra.run.smithi113.stdout: python3-ply-3.11-14.el9.noarch 2024-05-03T18:35:40.870 INFO:teuthology.orchestra.run.smithi113.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-03T18:35:40.870 INFO:teuthology.orchestra.run.smithi113.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-03T18:35:40.870 INFO:teuthology.orchestra.run.smithi113.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-03T18:35:40.870 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-03T18:35:40.870 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-03T18:35:40.870 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-03T18:35:40.871 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-03T18:35:40.871 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-03T18:35:40.871 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-03T18:35:40.871 INFO:teuthology.orchestra.run.smithi113.stdout: python3-rados-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:35:40.871 INFO:teuthology.orchestra.run.smithi113.stdout: python3-rbd-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:35:40.871 INFO:teuthology.orchestra.run.smithi113.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-03T18:35:40.871 INFO:teuthology.orchestra.run.smithi113.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-03T18:35:40.871 INFO:teuthology.orchestra.run.smithi113.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-03T18:35:40.871 INFO:teuthology.orchestra.run.smithi113.stdout: python3-rgw-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:35:40.871 INFO:teuthology.orchestra.run.smithi113.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-03T18:35:40.871 INFO:teuthology.orchestra.run.smithi113.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-03T18:35:40.871 INFO:teuthology.orchestra.run.smithi113.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-03T18:35:40.871 INFO:teuthology.orchestra.run.smithi113.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-03T18:35:40.872 INFO:teuthology.orchestra.run.smithi113.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-03T18:35:40.872 INFO:teuthology.orchestra.run.smithi113.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-03T18:35:40.872 INFO:teuthology.orchestra.run.smithi113.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-03T18:35:40.872 INFO:teuthology.orchestra.run.smithi113.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-03T18:35:40.872 INFO:teuthology.orchestra.run.smithi113.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-03T18:35:40.872 INFO:teuthology.orchestra.run.smithi113.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-03T18:35:40.872 INFO:teuthology.orchestra.run.smithi113.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-03T18:35:40.872 INFO:teuthology.orchestra.run.smithi113.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-03T18:35:40.872 INFO:teuthology.orchestra.run.smithi113.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-03T18:35:40.872 INFO:teuthology.orchestra.run.smithi113.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-03T18:35:40.872 INFO:teuthology.orchestra.run.smithi113.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-03T18:35:40.872 INFO:teuthology.orchestra.run.smithi113.stdout: rbd-fuse-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:35:40.872 INFO:teuthology.orchestra.run.smithi113.stdout: rbd-mirror-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:35:40.872 INFO:teuthology.orchestra.run.smithi113.stdout: rbd-nbd-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:35:40.872 INFO:teuthology.orchestra.run.smithi113.stdout: re2-1:20211101-3.el9.x86_64 2024-05-03T18:35:40.873 INFO:teuthology.orchestra.run.smithi113.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-03T18:35:40.873 INFO:teuthology.orchestra.run.smithi113.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-03T18:35:40.873 INFO:teuthology.orchestra.run.smithi113.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-03T18:35:40.873 INFO:teuthology.orchestra.run.smithi113.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-03T18:35:40.873 INFO:teuthology.orchestra.run.smithi113.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-03T18:35:40.873 INFO:teuthology.orchestra.run.smithi113.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-03T18:35:40.873 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-05-03T18:35:40.873 INFO:teuthology.orchestra.run.smithi113.stdout:Complete! 2024-05-03T18:35:41.256 DEBUG:teuthology.parallel:result is None 2024-05-03T18:35:42.385 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-05-03T18:35:42.385 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-2:19.0.0-3450.g9a047fe7.el9.x86_64 1/156 2024-05-03T18:35:42.385 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-base-2:19.0.0-3450.g9a047fe7.el9.x86_64 2/156 2024-05-03T18:35:42.386 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-common-2:19.0.0-3450.g9a047fe7.el9.x86_64 3/156 2024-05-03T18:35:42.386 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-fuse-2:19.0.0-3450.g9a047fe7.el9.x86_64 4/156 2024-05-03T18:35:42.386 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3450.g9a047 5/156 2024-05-03T18:35:42.386 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mds-2:19.0.0-3450.g9a047fe7.el9.x86_64 6/156 2024-05-03T18:35:42.386 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mgr-2:19.0.0-3450.g9a047fe7.el9.x86_64 7/156 2024-05-03T18:35:42.386 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mon-2:19.0.0-3450.g9a047fe7.el9.x86_64 8/156 2024-05-03T18:35:42.386 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-osd-2:19.0.0-3450.g9a047fe7.el9.x86_64 9/156 2024-05-03T18:35:42.386 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-radosgw-2:19.0.0-3450.g9a047fe7.el9.x86_64 10/156 2024-05-03T18:35:42.386 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-selinux-2:19.0.0-3450.g9a047fe7.el9.x86_64 11/156 2024-05-03T18:35:42.386 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-test-2:19.0.0-3450.g9a047fe7.el9.x86_64 12/156 2024-05-03T18:35:42.386 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : cephfs-mirror-2:19.0.0-3450.g9a047fe7.el9.x86_64 13/156 2024-05-03T18:35:42.387 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : cephfs-shell-2:19.0.0-3450.g9a047fe7.el9.x86_64 14/156 2024-05-03T18:35:42.387 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libcephfs-devel-2:19.0.0-3450.g9a047fe7.el9.x86_ 15/156 2024-05-03T18:35:42.387 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libcephfs2-2:19.0.0-3450.g9a047fe7.el9.x86_64 16/156 2024-05-03T18:35:42.387 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libcephsqlite-2:19.0.0-3450.g9a047fe7.el9.x86_64 17/156 2024-05-03T18:35:42.387 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librados-devel-2:19.0.0-3450.g9a047fe7.el9.x86_6 18/156 2024-05-03T18:35:42.387 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libradosstriper1-2:19.0.0-3450.g9a047fe7.el9.x86 19/156 2024-05-03T18:35:42.387 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librgw2-2:19.0.0-3450.g9a047fe7.el9.x86_64 20/156 2024-05-03T18:35:42.387 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3450.g9a047fe7.el 21/156 2024-05-03T18:35:42.388 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-ceph-common-2:19.0.0-3450.g9a047fe7.el9. 22/156 2024-05-03T18:35:42.388 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-cephfs-2:19.0.0-3450.g9a047fe7.el9.x86_6 23/156 2024-05-03T18:35:42.388 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-rados-2:19.0.0-3450.g9a047fe7.el9.x86_64 24/156 2024-05-03T18:35:42.388 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-rbd-2:19.0.0-3450.g9a047fe7.el9.x86_64 25/156 2024-05-03T18:35:42.388 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-rgw-2:19.0.0-3450.g9a047fe7.el9.x86_64 26/156 2024-05-03T18:35:42.388 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : rbd-fuse-2:19.0.0-3450.g9a047fe7.el9.x86_64 27/156 2024-05-03T18:35:42.389 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : rbd-mirror-2:19.0.0-3450.g9a047fe7.el9.x86_64 28/156 2024-05-03T18:35:42.389 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : rbd-nbd-2:19.0.0-3450.g9a047fe7.el9.x86_64 29/156 2024-05-03T18:35:42.400 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3450.g9a047fe7. 30/156 2024-05-03T18:35:42.400 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3450.g9a047fe7.el9.noa 31/156 2024-05-03T18:35:42.401 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3450.g9a047fe7.el9.n 32/156 2024-05-03T18:35:42.401 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3450.g9a0 33/156 2024-05-03T18:35:42.401 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3450.g9a047fe7.el 34/156 2024-05-03T18:35:42.401 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3450.g9a047fe7.el9.noarch 35/156 2024-05-03T18:35:42.401 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3450.g9a047fe7.e 36/156 2024-05-03T18:35:42.401 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-volume-2:19.0.0-3450.g9a047fe7.el9.noarch 37/156 2024-05-03T18:35:42.401 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : cephadm-2:19.0.0-3450.g9a047fe7.el9.noarch 38/156 2024-05-03T18:35:42.401 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : cephfs-top-2:19.0.0-3450.g9a047fe7.el9.noarch 39/156 2024-05-03T18:35:42.401 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-05-03T18:35:42.401 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-05-03T18:35:42.401 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-05-03T18:35:42.401 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-05-03T18:35:42.401 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-05-03T18:35:42.402 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-05-03T18:35:42.402 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-05-03T18:35:42.402 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-05-03T18:35:42.402 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-05-03T18:35:42.402 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-05-03T18:35:42.402 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-05-03T18:35:42.402 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-05-03T18:35:42.402 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-05-03T18:35:42.402 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-05-03T18:35:42.402 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-05-03T18:35:42.402 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-05-03T18:35:42.402 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-05-03T18:35:42.402 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-05-03T18:35:42.402 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-05-03T18:35:42.403 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-05-03T18:35:42.403 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-05-03T18:35:42.403 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-05-03T18:35:42.403 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-05-03T18:35:42.403 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-05-03T18:35:42.403 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-05-03T18:35:42.403 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-05-03T18:35:42.403 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-05-03T18:35:42.403 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-05-03T18:35:42.403 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-05-03T18:35:42.403 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-05-03T18:35:42.403 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-05-03T18:35:42.403 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-05-03T18:35:42.403 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-05-03T18:35:42.404 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-05-03T18:35:42.404 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-05-03T18:35:42.404 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-05-03T18:35:42.404 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-05-03T18:35:42.404 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-05-03T18:35:42.404 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-05-03T18:35:42.404 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-05-03T18:35:42.404 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-05-03T18:35:42.404 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-05-03T18:35:42.404 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-05-03T18:35:42.404 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-05-03T18:35:42.404 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-05-03T18:35:42.404 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-05-03T18:35:42.404 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-05-03T18:35:42.404 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-05-03T18:35:42.405 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-05-03T18:35:42.405 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-05-03T18:35:42.405 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-05-03T18:35:42.405 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-05-03T18:35:42.405 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-05-03T18:35:42.405 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-05-03T18:35:42.405 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-05-03T18:35:42.405 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-05-03T18:35:42.405 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-05-03T18:35:42.405 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-05-03T18:35:42.405 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-05-03T18:35:42.405 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-05-03T18:35:42.405 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-05-03T18:35:42.405 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-05-03T18:35:42.405 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-05-03T18:35:42.406 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-05-03T18:35:42.406 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-05-03T18:35:42.406 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-05-03T18:35:42.406 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-05-03T18:35:42.406 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-05-03T18:35:42.406 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/156 2024-05-03T18:35:42.406 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/156 2024-05-03T18:35:42.406 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-05-03T18:35:42.406 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-05-03T18:35:42.406 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-05-03T18:35:42.406 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/156 2024-05-03T18:35:42.406 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-05-03T18:35:42.406 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-05-03T18:35:42.406 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-05-03T18:35:42.406 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-05-03T18:35:42.407 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-05-03T18:35:42.407 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-05-03T18:35:42.407 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-05-03T18:35:42.407 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-05-03T18:35:42.407 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-05-03T18:35:42.407 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-05-03T18:35:42.407 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-05-03T18:35:42.407 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-05-03T18:35:42.407 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-05-03T18:35:42.407 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-05-03T18:35:42.407 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-05-03T18:35:42.407 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-05-03T18:35:42.407 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-05-03T18:35:42.407 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-05-03T18:35:42.407 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-05-03T18:35:42.408 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-05-03T18:35:42.408 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-05-03T18:35:42.408 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-05-03T18:35:42.408 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-05-03T18:35:42.408 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-05-03T18:35:42.408 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-05-03T18:35:42.408 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-05-03T18:35:42.408 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-05-03T18:35:42.408 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-05-03T18:35:42.408 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-05-03T18:35:42.408 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-05-03T18:35:42.408 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-05-03T18:35:42.408 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-05-03T18:35:42.408 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-05-03T18:35:42.408 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-05-03T18:35:42.408 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-05-03T18:35:42.409 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librados2-2:19.0.0-3450.g9a047fe7.el9.x86_64 149/156 2024-05-03T18:35:42.409 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-05-03T18:35:42.409 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librbd1-2:19.0.0-3450.g9a047fe7.el9.x86_64 151/156 2024-05-03T18:35:42.409 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-05-03T18:35:42.409 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-05-03T18:35:42.409 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-05-03T18:35:42.409 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-05-03T18:35:44.514 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-05-03T18:35:44.514 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-03T18:35:44.514 INFO:teuthology.orchestra.run.smithi045.stdout:Upgraded: 2024-05-03T18:35:44.514 INFO:teuthology.orchestra.run.smithi045.stdout: librados2-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:35:44.514 INFO:teuthology.orchestra.run.smithi045.stdout: librbd1-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:35:44.515 INFO:teuthology.orchestra.run.smithi045.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-03T18:35:44.515 INFO:teuthology.orchestra.run.smithi045.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-03T18:35:44.515 INFO:teuthology.orchestra.run.smithi045.stdout:Installed: 2024-05-03T18:35:44.515 INFO:teuthology.orchestra.run.smithi045.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-03T18:35:44.515 INFO:teuthology.orchestra.run.smithi045.stdout: automake-1.16.2-8.el9.noarch 2024-05-03T18:35:44.515 INFO:teuthology.orchestra.run.smithi045.stdout: bison-3.7.4-5.el9.x86_64 2024-05-03T18:35:44.515 INFO:teuthology.orchestra.run.smithi045.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-03T18:35:44.515 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:35:44.515 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-base-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:35:44.515 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-common-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:35:44.515 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-fuse-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:35:44.515 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-grafana-dashboards-2:19.0.0-3450.g9a047fe7.el9.noarch 2024-05-03T18:35:44.515 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-immutable-object-cache-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:35:44.516 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mds-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:35:44.516 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:35:44.516 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-cephadm-2:19.0.0-3450.g9a047fe7.el9.noarch 2024-05-03T18:35:44.516 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-dashboard-2:19.0.0-3450.g9a047fe7.el9.noarch 2024-05-03T18:35:44.516 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3450.g9a047fe7.el9.noarch 2024-05-03T18:35:44.516 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-modules-core-2:19.0.0-3450.g9a047fe7.el9.noarch 2024-05-03T18:35:44.516 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-rook-2:19.0.0-3450.g9a047fe7.el9.noarch 2024-05-03T18:35:44.516 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mon-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:35:44.516 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-osd-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:35:44.516 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-prometheus-alerts-2:19.0.0-3450.g9a047fe7.el9.noarch 2024-05-03T18:35:44.516 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-radosgw-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:35:44.516 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-selinux-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:35:44.516 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-test-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:35:44.516 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-volume-2:19.0.0-3450.g9a047fe7.el9.noarch 2024-05-03T18:35:44.517 INFO:teuthology.orchestra.run.smithi045.stdout: cephadm-2:19.0.0-3450.g9a047fe7.el9.noarch 2024-05-03T18:35:44.517 INFO:teuthology.orchestra.run.smithi045.stdout: cephfs-mirror-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:35:44.517 INFO:teuthology.orchestra.run.smithi045.stdout: cephfs-shell-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:35:44.517 INFO:teuthology.orchestra.run.smithi045.stdout: cephfs-top-2:19.0.0-3450.g9a047fe7.el9.noarch 2024-05-03T18:35:44.517 INFO:teuthology.orchestra.run.smithi045.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-05-03T18:35:44.517 INFO:teuthology.orchestra.run.smithi045.stdout: flex-2.6.4-9.el9.x86_64 2024-05-03T18:35:44.517 INFO:teuthology.orchestra.run.smithi045.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-03T18:35:44.517 INFO:teuthology.orchestra.run.smithi045.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-03T18:35:44.517 INFO:teuthology.orchestra.run.smithi045.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-03T18:35:44.517 INFO:teuthology.orchestra.run.smithi045.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-03T18:35:44.517 INFO:teuthology.orchestra.run.smithi045.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-03T18:35:44.517 INFO:teuthology.orchestra.run.smithi045.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-03T18:35:44.517 INFO:teuthology.orchestra.run.smithi045.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-03T18:35:44.518 INFO:teuthology.orchestra.run.smithi045.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-03T18:35:44.518 INFO:teuthology.orchestra.run.smithi045.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-03T18:35:44.518 INFO:teuthology.orchestra.run.smithi045.stdout: libcephfs-devel-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:35:44.518 INFO:teuthology.orchestra.run.smithi045.stdout: libcephfs2-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:35:44.518 INFO:teuthology.orchestra.run.smithi045.stdout: libcephsqlite-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:35:44.518 INFO:teuthology.orchestra.run.smithi045.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-03T18:35:44.518 INFO:teuthology.orchestra.run.smithi045.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-03T18:35:44.518 INFO:teuthology.orchestra.run.smithi045.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-03T18:35:44.518 INFO:teuthology.orchestra.run.smithi045.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-03T18:35:44.518 INFO:teuthology.orchestra.run.smithi045.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-03T18:35:44.518 INFO:teuthology.orchestra.run.smithi045.stdout: librados-devel-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:35:44.518 INFO:teuthology.orchestra.run.smithi045.stdout: libradosstriper1-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:35:44.518 INFO:teuthology.orchestra.run.smithi045.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-03T18:35:44.519 INFO:teuthology.orchestra.run.smithi045.stdout: librgw2-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:35:44.519 INFO:teuthology.orchestra.run.smithi045.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-03T18:35:44.519 INFO:teuthology.orchestra.run.smithi045.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-03T18:35:44.519 INFO:teuthology.orchestra.run.smithi045.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-03T18:35:44.519 INFO:teuthology.orchestra.run.smithi045.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-03T18:35:44.519 INFO:teuthology.orchestra.run.smithi045.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-03T18:35:44.519 INFO:teuthology.orchestra.run.smithi045.stdout: lua-5.4.4-4.el9.x86_64 2024-05-03T18:35:44.519 INFO:teuthology.orchestra.run.smithi045.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-03T18:35:44.519 INFO:teuthology.orchestra.run.smithi045.stdout: luarocks-3.9.2-1.el9.noarch 2024-05-03T18:35:44.519 INFO:teuthology.orchestra.run.smithi045.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-03T18:35:44.519 INFO:teuthology.orchestra.run.smithi045.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-03T18:35:44.519 INFO:teuthology.orchestra.run.smithi045.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-03T18:35:44.519 INFO:teuthology.orchestra.run.smithi045.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-03T18:35:44.519 INFO:teuthology.orchestra.run.smithi045.stdout: postgresql-13.14-1.el9.x86_64 2024-05-03T18:35:44.519 INFO:teuthology.orchestra.run.smithi045.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-03T18:35:44.520 INFO:teuthology.orchestra.run.smithi045.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-03T18:35:44.520 INFO:teuthology.orchestra.run.smithi045.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-03T18:35:44.520 INFO:teuthology.orchestra.run.smithi045.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-03T18:35:44.520 INFO:teuthology.orchestra.run.smithi045.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-03T18:35:44.520 INFO:teuthology.orchestra.run.smithi045.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-03T18:35:44.520 INFO:teuthology.orchestra.run.smithi045.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-03T18:35:44.520 INFO:teuthology.orchestra.run.smithi045.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-03T18:35:44.520 INFO:teuthology.orchestra.run.smithi045.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-03T18:35:44.520 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-03T18:35:44.520 INFO:teuthology.orchestra.run.smithi045.stdout: python3-ceph-argparse-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:35:44.520 INFO:teuthology.orchestra.run.smithi045.stdout: python3-ceph-common-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:35:44.520 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cephfs-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:35:44.520 INFO:teuthology.orchestra.run.smithi045.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-03T18:35:44.521 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-03T18:35:44.521 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-03T18:35:44.521 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-03T18:35:44.521 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-03T18:35:44.521 INFO:teuthology.orchestra.run.smithi045.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-05-03T18:35:44.521 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-03T18:35:44.521 INFO:teuthology.orchestra.run.smithi045.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-03T18:35:44.521 INFO:teuthology.orchestra.run.smithi045.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-03T18:35:44.521 INFO:teuthology.orchestra.run.smithi045.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-03T18:35:44.521 INFO:teuthology.orchestra.run.smithi045.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-03T18:35:44.521 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-03T18:35:44.521 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-03T18:35:44.521 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-03T18:35:44.521 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-03T18:35:44.522 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-03T18:35:44.522 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-03T18:35:44.522 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-03T18:35:44.522 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-03T18:35:44.522 INFO:teuthology.orchestra.run.smithi045.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-03T18:35:44.522 INFO:teuthology.orchestra.run.smithi045.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-03T18:35:44.522 INFO:teuthology.orchestra.run.smithi045.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-03T18:35:44.522 INFO:teuthology.orchestra.run.smithi045.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-03T18:35:44.522 INFO:teuthology.orchestra.run.smithi045.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-03T18:35:44.522 INFO:teuthology.orchestra.run.smithi045.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-03T18:35:44.522 INFO:teuthology.orchestra.run.smithi045.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-03T18:35:44.522 INFO:teuthology.orchestra.run.smithi045.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-03T18:35:44.522 INFO:teuthology.orchestra.run.smithi045.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-03T18:35:44.523 INFO:teuthology.orchestra.run.smithi045.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-03T18:35:44.523 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-03T18:35:44.523 INFO:teuthology.orchestra.run.smithi045.stdout: python3-ply-3.11-14.el9.noarch 2024-05-03T18:35:44.523 INFO:teuthology.orchestra.run.smithi045.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-03T18:35:44.523 INFO:teuthology.orchestra.run.smithi045.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-03T18:35:44.523 INFO:teuthology.orchestra.run.smithi045.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-03T18:35:44.523 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-03T18:35:44.523 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-03T18:35:44.523 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-03T18:35:44.523 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-03T18:35:44.523 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-03T18:35:44.523 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-03T18:35:44.523 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rados-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:35:44.524 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rbd-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:35:44.524 INFO:teuthology.orchestra.run.smithi045.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-03T18:35:44.524 INFO:teuthology.orchestra.run.smithi045.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-03T18:35:44.524 INFO:teuthology.orchestra.run.smithi045.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-03T18:35:44.524 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rgw-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:35:44.524 INFO:teuthology.orchestra.run.smithi045.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-03T18:35:44.524 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-03T18:35:44.524 INFO:teuthology.orchestra.run.smithi045.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-03T18:35:44.524 INFO:teuthology.orchestra.run.smithi045.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-03T18:35:44.524 INFO:teuthology.orchestra.run.smithi045.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-03T18:35:44.524 INFO:teuthology.orchestra.run.smithi045.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-03T18:35:44.524 INFO:teuthology.orchestra.run.smithi045.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-03T18:35:44.524 INFO:teuthology.orchestra.run.smithi045.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-03T18:35:44.524 INFO:teuthology.orchestra.run.smithi045.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-03T18:35:44.525 INFO:teuthology.orchestra.run.smithi045.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-03T18:35:44.525 INFO:teuthology.orchestra.run.smithi045.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-03T18:35:44.525 INFO:teuthology.orchestra.run.smithi045.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-03T18:35:44.525 INFO:teuthology.orchestra.run.smithi045.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-03T18:35:44.525 INFO:teuthology.orchestra.run.smithi045.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-03T18:35:44.525 INFO:teuthology.orchestra.run.smithi045.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-03T18:35:44.525 INFO:teuthology.orchestra.run.smithi045.stdout: rbd-fuse-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:35:44.525 INFO:teuthology.orchestra.run.smithi045.stdout: rbd-mirror-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:35:44.525 INFO:teuthology.orchestra.run.smithi045.stdout: rbd-nbd-2:19.0.0-3450.g9a047fe7.el9.x86_64 2024-05-03T18:35:44.525 INFO:teuthology.orchestra.run.smithi045.stdout: re2-1:20211101-3.el9.x86_64 2024-05-03T18:35:44.525 INFO:teuthology.orchestra.run.smithi045.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-03T18:35:44.525 INFO:teuthology.orchestra.run.smithi045.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-03T18:35:44.525 INFO:teuthology.orchestra.run.smithi045.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-03T18:35:44.525 INFO:teuthology.orchestra.run.smithi045.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-03T18:35:44.526 INFO:teuthology.orchestra.run.smithi045.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-03T18:35:44.526 INFO:teuthology.orchestra.run.smithi045.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-03T18:35:44.526 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-05-03T18:35:44.526 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2024-05-03T18:35:44.914 DEBUG:teuthology.parallel:result is None 2024-05-03T18:35:44.915 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=9a047fe7599a549042d06a3fb5b6813a203eabea 2024-05-03T18:35:45.145 DEBUG:teuthology.orchestra.run.smithi032:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-03T18:35:45.171 INFO:teuthology.orchestra.run.smithi032.stdout:19.0.0-3450.g9a047fe7.el9 2024-05-03T18:35:45.172 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3450.g9a047fe7.el9 2024-05-03T18:35:45.172 INFO:teuthology.task.install:The correct ceph version 19.0.0-3450.g9a047fe7 is installed. 2024-05-03T18:35:45.174 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=9a047fe7599a549042d06a3fb5b6813a203eabea 2024-05-03T18:35:45.387 DEBUG:teuthology.orchestra.run.smithi045:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-03T18:35:45.411 INFO:teuthology.orchestra.run.smithi045.stdout:19.0.0-3450.g9a047fe7.el9 2024-05-03T18:35:45.411 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3450.g9a047fe7.el9 2024-05-03T18:35:45.411 INFO:teuthology.task.install:The correct ceph version 19.0.0-3450.g9a047fe7 is installed. 2024-05-03T18:35:45.413 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=9a047fe7599a549042d06a3fb5b6813a203eabea 2024-05-03T18:35:45.710 DEBUG:teuthology.orchestra.run.smithi113:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-03T18:35:45.736 INFO:teuthology.orchestra.run.smithi113.stdout:19.0.0-3450.g9a047fe7.el9 2024-05-03T18:35:45.736 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3450.g9a047fe7.el9 2024-05-03T18:35:45.736 INFO:teuthology.task.install:The correct ceph version 19.0.0-3450.g9a047fe7 is installed. 2024-05-03T18:35:45.739 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-05-03T18:35:45.739 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-05-03T18:35:45.739 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-03T18:35:45.771 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-05-03T18:35:45.771 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-03T18:35:45.798 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-05-03T18:35:45.798 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-03T18:35:45.830 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-05-03T18:35:45.830 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-05-03T18:35:45.830 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd of=/usr/bin/daemon-helper 2024-05-03T18:35:45.860 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-03T18:35:45.930 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-05-03T18:35:45.930 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/usr/bin/daemon-helper 2024-05-03T18:35:45.956 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-03T18:35:46.022 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-05-03T18:35:46.022 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd of=/usr/bin/daemon-helper 2024-05-03T18:35:46.050 DEBUG:teuthology.orchestra.run.smithi113:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-03T18:35:46.118 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-05-03T18:35:46.118 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-05-03T18:35:46.118 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-03T18:35:46.144 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-03T18:35:46.209 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-05-03T18:35:46.209 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-03T18:35:46.234 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-03T18:35:46.301 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-05-03T18:35:46.302 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-03T18:35:46.329 DEBUG:teuthology.orchestra.run.smithi113:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-03T18:35:46.397 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-05-03T18:35:46.397 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-05-03T18:35:46.397 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd of=/usr/bin/stdin-killer 2024-05-03T18:35:46.422 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-03T18:35:46.486 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-05-03T18:35:46.486 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/usr/bin/stdin-killer 2024-05-03T18:35:46.512 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-03T18:35:46.576 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-05-03T18:35:46.576 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd of=/usr/bin/stdin-killer 2024-05-03T18:35:46.604 DEBUG:teuthology.orchestra.run.smithi113:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-03T18:35:46.671 INFO:teuthology.run_tasks:Running task cephadm... 2024-05-03T18:35:46.782 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'mds_export_ephemeral_random_max': 0.1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'sha1': '9a047fe7599a549042d06a3fb5b6813a203eabea', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-05-03T18:35:46.783 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea 2024-05-03T18:35:46.783 INFO:tasks.cephadm:Cluster fsid is f500455e-097b-11ef-bc97-c7b262605968 2024-05-03T18:35:46.784 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-05-03T18:35:46.784 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.32', 'mon.b': '172.21.15.45', 'mon.c': '172.21.15.113'} 2024-05-03T18:35:46.784 INFO:tasks.cephadm:First mon is mon.a on smithi032 2024-05-03T18:35:46.784 INFO:tasks.cephadm:First mgr is x 2024-05-03T18:35:46.784 INFO:tasks.cephadm:Normalizing hostnames... 2024-05-03T18:35:46.784 DEBUG:teuthology.orchestra.run.smithi032:> sudo hostname $(hostname -s) 2024-05-03T18:35:46.809 DEBUG:teuthology.orchestra.run.smithi045:> sudo hostname $(hostname -s) 2024-05-03T18:35:46.834 DEBUG:teuthology.orchestra.run.smithi113:> sudo hostname $(hostname -s) 2024-05-03T18:35:46.861 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-05-03T18:35:46.862 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=9a047fe7599a549042d06a3fb5b6813a203eabea 2024-05-03T18:35:47.003 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '9a047fe7599a549042d06a3fb5b6813a203eabea', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/79034/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-3450-g9a047fe7', 'node_name': '172.21.5.34+adami04', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-3450.g9a047fe7'}, 'url': 'https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240501.193033/9a047fe7599a549042d06a3fb5b6813a203eabea/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-05-02 00:36:37.664792', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-rishabh-testing-20240501.193033', 'chacra_url': 'https://4.chacra.ceph.com/repos/ceph/wip-rishabh-testing-20240501.193033/9a047fe7599a549042d06a3fb5b6813a203eabea/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-05-03T18:35:47.223 INFO:tasks.util.chacra:got chacra host 4.chacra.ceph.com, ref wip-rishabh-testing-20240501.193033, sha1 9a047fe7599a549042d06a3fb5b6813a203eabea from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=9a047fe7599a549042d06a3fb5b6813a203eabea 2024-05-03T18:35:47.225 INFO:tasks.cephadm:Discovered cachra url: https://4.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240501.193033/9a047fe7599a549042d06a3fb5b6813a203eabea/centos/9/x86_64/flavors/default/cephadm 2024-05-03T18:35:47.225 DEBUG:teuthology.orchestra.run.smithi032:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240501.193033/9a047fe7599a549042d06a3fb5b6813a203eabea/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-03T18:35:48.241 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r--. 1 ubuntu ubuntu 782467 May 3 18:35 /home/ubuntu/cephtest/cephadm 2024-05-03T18:35:48.242 DEBUG:teuthology.orchestra.run.smithi045:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240501.193033/9a047fe7599a549042d06a3fb5b6813a203eabea/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-03T18:35:49.075 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r--. 1 ubuntu ubuntu 782467 May 3 18:35 /home/ubuntu/cephtest/cephadm 2024-05-03T18:35:49.075 DEBUG:teuthology.orchestra.run.smithi113:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240501.193033/9a047fe7599a549042d06a3fb5b6813a203eabea/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-03T18:35:50.607 INFO:teuthology.orchestra.run.smithi113.stdout:-rw-r--r--. 1 ubuntu ubuntu 782467 May 3 18:35 /home/ubuntu/cephtest/cephadm 2024-05-03T18:35:50.607 DEBUG:teuthology.orchestra.run.smithi032:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-03T18:35:50.627 DEBUG:teuthology.orchestra.run.smithi045:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-03T18:35:50.646 DEBUG:teuthology.orchestra.run.smithi113:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-03T18:35:50.681 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea on all hosts... 2024-05-03T18:35:50.681 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea pull 2024-05-03T18:35:50.684 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea pull 2024-05-03T18:35:50.689 DEBUG:teuthology.orchestra.run.smithi113:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea pull 2024-05-03T18:35:52.155 INFO:teuthology.orchestra.run.smithi032.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea... 2024-05-03T18:35:52.305 INFO:teuthology.orchestra.run.smithi045.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea... 2024-05-03T18:35:52.408 INFO:teuthology.orchestra.run.smithi113.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea... 2024-05-03T18:36:33.555 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-05-03T18:36:33.556 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_version": "ceph version 19.0.0-3450-g9a047fe7 (9a047fe7599a549042d06a3fb5b6813a203eabea) squid (dev)", 2024-05-03T18:36:33.556 INFO:teuthology.orchestra.run.smithi045.stdout: "image_id": "ee4d846202496ba6096593e8e8becc0700a5eb7ebd95536031187f5f337d4012", 2024-05-03T18:36:33.556 INFO:teuthology.orchestra.run.smithi045.stdout: "repo_digests": [ 2024-05-03T18:36:33.556 INFO:teuthology.orchestra.run.smithi045.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd" 2024-05-03T18:36:33.556 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-05-03T18:36:33.556 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-05-03T18:36:34.044 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-05-03T18:36:34.044 INFO:teuthology.orchestra.run.smithi113.stdout: "ceph_version": "ceph version 19.0.0-3450-g9a047fe7 (9a047fe7599a549042d06a3fb5b6813a203eabea) squid (dev)", 2024-05-03T18:36:34.044 INFO:teuthology.orchestra.run.smithi113.stdout: "image_id": "ee4d846202496ba6096593e8e8becc0700a5eb7ebd95536031187f5f337d4012", 2024-05-03T18:36:34.044 INFO:teuthology.orchestra.run.smithi113.stdout: "repo_digests": [ 2024-05-03T18:36:34.044 INFO:teuthology.orchestra.run.smithi113.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd" 2024-05-03T18:36:34.044 INFO:teuthology.orchestra.run.smithi113.stdout: ] 2024-05-03T18:36:34.044 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-05-03T18:36:34.695 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-05-03T18:36:34.695 INFO:teuthology.orchestra.run.smithi032.stdout: "ceph_version": "ceph version 19.0.0-3450-g9a047fe7 (9a047fe7599a549042d06a3fb5b6813a203eabea) squid (dev)", 2024-05-03T18:36:34.695 INFO:teuthology.orchestra.run.smithi032.stdout: "image_id": "ee4d846202496ba6096593e8e8becc0700a5eb7ebd95536031187f5f337d4012", 2024-05-03T18:36:34.695 INFO:teuthology.orchestra.run.smithi032.stdout: "repo_digests": [ 2024-05-03T18:36:34.695 INFO:teuthology.orchestra.run.smithi032.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd" 2024-05-03T18:36:34.695 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2024-05-03T18:36:34.695 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-05-03T18:36:34.719 DEBUG:teuthology.orchestra.run.smithi032:> sudo mkdir -p /etc/ceph 2024-05-03T18:36:34.772 DEBUG:teuthology.orchestra.run.smithi045:> sudo mkdir -p /etc/ceph 2024-05-03T18:36:34.805 DEBUG:teuthology.orchestra.run.smithi113:> sudo mkdir -p /etc/ceph 2024-05-03T18:36:34.833 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 777 /etc/ceph 2024-05-03T18:36:34.857 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 777 /etc/ceph 2024-05-03T18:36:34.886 DEBUG:teuthology.orchestra.run.smithi113:> sudo chmod 777 /etc/ceph 2024-05-03T18:36:34.910 INFO:tasks.cephadm:Writing seed config... 2024-05-03T18:36:34.911 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-05-03T18:36:34.911 INFO:tasks.cephadm: override: [client] debug client = 20 2024-05-03T18:36:34.911 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-05-03T18:36:34.912 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-05-03T18:36:34.912 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-05-03T18:36:34.912 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-05-03T18:36:34.912 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-05-03T18:36:34.912 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-05-03T18:36:34.912 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-05-03T18:36:34.912 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-05-03T18:36:34.912 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-05-03T18:36:34.912 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-05-03T18:36:34.912 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-05-03T18:36:34.912 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-05-03T18:36:34.912 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-05-03T18:36:34.912 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-05-03T18:36:34.912 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-05-03T18:36:34.912 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2024-05-03T18:36:34.912 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-05-03T18:36:34.912 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-05-03T18:36:34.913 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-05-03T18:36:34.913 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-05-03T18:36:34.913 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-05-03T18:36:34.913 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-05-03T18:36:34.913 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-05-03T18:36:34.913 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-05-03T18:36:34.913 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-05-03T18:36:34.913 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-05-03T18:36:34.913 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-05-03T18:36:34.913 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-05-03T18:36:34.913 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-05-03T18:36:34.913 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2024-05-03T18:36:34.913 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2024-05-03T18:36:34.913 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2024-05-03T18:36:34.913 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-05-03T18:36:34.913 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-05-03T18:36:34.914 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2024-05-03T18:36:34.914 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2024-05-03T18:36:34.914 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-05-03T18:36:34.914 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-05-03T18:36:34.914 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2024-05-03T18:36:34.914 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-05-03T18:36:34.914 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-05-03T18:36:34.914 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-05-03T18:36:34.914 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-05-03T18:36:34.914 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-05-03T18:36:34.914 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-05-03T18:36:34.914 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-05-03T18:36:34.914 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2024-05-03T18:36:34.915 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-05-03T18:36:34.915 DEBUG:teuthology.orchestra.run.smithi032:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-05-03T18:36:34.930 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = f500455e-097b-11ef-bc97-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_bal_replicate_threshold = 1 mds_export_ephemeral_random_max = 0.1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-05-03T18:36:34.931 DEBUG:teuthology.orchestra.run.smithi032:mon.a> sudo journalctl -f -n 0 -u ceph-f500455e-097b-11ef-bc97-c7b262605968@mon.a.service 2024-05-03T18:36:34.973 DEBUG:teuthology.orchestra.run.smithi032:mgr.x> sudo journalctl -f -n 0 -u ceph-f500455e-097b-11ef-bc97-c7b262605968@mgr.x.service 2024-05-03T18:36:35.015 INFO:tasks.cephadm:Bootstrapping... 2024-05-03T18:36:35.016 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea -v bootstrap --fsid f500455e-097b-11ef-bc97-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.32 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-05-03T18:36:35.212 INFO:teuthology.orchestra.run.smithi032.stdout:-------------------------------------------------------------------------------- 2024-05-03T18:36:35.212 INFO:teuthology.orchestra.run.smithi032.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea', '-v', 'bootstrap', '--fsid', 'f500455e-097b-11ef-bc97-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.32', '--skip-admin-label'] 2024-05-03T18:36:35.213 INFO:teuthology.orchestra.run.smithi032.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-05-03T18:36:35.213 INFO:teuthology.orchestra.run.smithi032.stdout:Verifying podman|docker is present... 2024-05-03T18:36:35.247 INFO:teuthology.orchestra.run.smithi032.stdout:/bin/podman: stdout 4.9.4-dev 2024-05-03T18:36:35.247 INFO:teuthology.orchestra.run.smithi032.stdout:Verifying lvm2 is present... 2024-05-03T18:36:35.247 INFO:teuthology.orchestra.run.smithi032.stdout:Verifying time synchronization is in place... 2024-05-03T18:36:35.255 INFO:teuthology.orchestra.run.smithi032.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-05-03T18:36:35.255 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-05-03T18:36:35.261 INFO:teuthology.orchestra.run.smithi032.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-05-03T18:36:35.261 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stdout inactive 2024-05-03T18:36:35.268 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stdout enabled 2024-05-03T18:36:35.274 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stdout active 2024-05-03T18:36:35.274 INFO:teuthology.orchestra.run.smithi032.stdout:Unit chronyd.service is enabled and running 2024-05-03T18:36:35.274 INFO:teuthology.orchestra.run.smithi032.stdout:Repeating the final host check... 2024-05-03T18:36:35.297 INFO:teuthology.orchestra.run.smithi032.stdout:/bin/podman: stdout 4.9.4-dev 2024-05-03T18:36:35.297 INFO:teuthology.orchestra.run.smithi032.stdout:podman (/bin/podman) version 4.9.4 is present 2024-05-03T18:36:35.297 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl is present 2024-05-03T18:36:35.297 INFO:teuthology.orchestra.run.smithi032.stdout:lvcreate is present 2024-05-03T18:36:35.304 INFO:teuthology.orchestra.run.smithi032.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-05-03T18:36:35.304 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-05-03T18:36:35.310 INFO:teuthology.orchestra.run.smithi032.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-05-03T18:36:35.310 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stdout inactive 2024-05-03T18:36:35.316 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stdout enabled 2024-05-03T18:36:35.323 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stdout active 2024-05-03T18:36:35.323 INFO:teuthology.orchestra.run.smithi032.stdout:Unit chronyd.service is enabled and running 2024-05-03T18:36:35.323 INFO:teuthology.orchestra.run.smithi032.stdout:Host looks OK 2024-05-03T18:36:35.323 INFO:teuthology.orchestra.run.smithi032.stdout:Cluster fsid: f500455e-097b-11ef-bc97-c7b262605968 2024-05-03T18:36:35.323 INFO:teuthology.orchestra.run.smithi032.stdout:Acquiring lock 139877680422192 on /run/cephadm/f500455e-097b-11ef-bc97-c7b262605968.lock 2024-05-03T18:36:35.323 INFO:teuthology.orchestra.run.smithi032.stdout:Lock 139877680422192 acquired on /run/cephadm/f500455e-097b-11ef-bc97-c7b262605968.lock 2024-05-03T18:36:35.324 INFO:teuthology.orchestra.run.smithi032.stdout:Verifying IP 172.21.15.32 port 3300 ... 2024-05-03T18:36:35.324 INFO:teuthology.orchestra.run.smithi032.stdout:Verifying IP 172.21.15.32 port 6789 ... 2024-05-03T18:36:35.324 INFO:teuthology.orchestra.run.smithi032.stdout:Base mon IP(s) is [172.21.15.32:3300, 172.21.15.32:6789], mon addrv is [v2:172.21.15.32:3300,v1:172.21.15.32:6789] 2024-05-03T18:36:35.328 INFO:teuthology.orchestra.run.smithi032.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.32 metric 100 2024-05-03T18:36:35.328 INFO:teuthology.orchestra.run.smithi032.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.32 metric 100 2024-05-03T18:36:35.330 INFO:teuthology.orchestra.run.smithi032.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-05-03T18:36:35.330 INFO:teuthology.orchestra.run.smithi032.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 1024 pref medium 2024-05-03T18:36:35.330 INFO:teuthology.orchestra.run.smithi032.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 100 pref medium 2024-05-03T18:36:35.333 INFO:teuthology.orchestra.run.smithi032.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-05-03T18:36:35.333 INFO:teuthology.orchestra.run.smithi032.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-05-03T18:36:35.333 INFO:teuthology.orchestra.run.smithi032.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-05-03T18:36:35.333 INFO:teuthology.orchestra.run.smithi032.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-05-03T18:36:35.333 INFO:teuthology.orchestra.run.smithi032.stdout:/sbin/ip: stdout inet6 fe80::ae1f:6bff:fef8:2602/64 scope link noprefixroute 2024-05-03T18:36:35.333 INFO:teuthology.orchestra.run.smithi032.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-05-03T18:36:35.334 INFO:teuthology.orchestra.run.smithi032.stdout:Mon IP `172.21.15.32` is in CIDR network `172.21.0.0/20` 2024-05-03T18:36:35.334 INFO:teuthology.orchestra.run.smithi032.stdout:Mon IP `172.21.15.32` is in CIDR network `172.21.0.0/20` 2024-05-03T18:36:35.334 INFO:teuthology.orchestra.run.smithi032.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-05-03T18:36:35.334 INFO:teuthology.orchestra.run.smithi032.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-05-03T18:36:35.335 INFO:teuthology.orchestra.run.smithi032.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea... 2024-05-03T18:36:35.806 INFO:teuthology.orchestra.run.smithi032.stdout:/bin/podman: stdout ee4d846202496ba6096593e8e8becc0700a5eb7ebd95536031187f5f337d4012 2024-05-03T18:36:35.806 INFO:teuthology.orchestra.run.smithi032.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea... 2024-05-03T18:36:35.806 INFO:teuthology.orchestra.run.smithi032.stdout:/bin/podman: stderr Getting image source signatures 2024-05-03T18:36:35.806 INFO:teuthology.orchestra.run.smithi032.stdout:/bin/podman: stderr Copying blob sha256:8173fd91c96404ae0424bb08037c57749051dcbdfaeee9354a65e201dad7b0e2 2024-05-03T18:36:35.806 INFO:teuthology.orchestra.run.smithi032.stdout:/bin/podman: stderr Copying blob sha256:e2f2fd2adca92139da4b4b907fa90a6775d3eb4eddaa977c42ef37c66eea70bd 2024-05-03T18:36:35.807 INFO:teuthology.orchestra.run.smithi032.stdout:/bin/podman: stderr Copying config sha256:ee4d846202496ba6096593e8e8becc0700a5eb7ebd95536031187f5f337d4012 2024-05-03T18:36:35.807 INFO:teuthology.orchestra.run.smithi032.stdout:/bin/podman: stderr Writing manifest to image destination 2024-05-03T18:36:36.721 INFO:teuthology.orchestra.run.smithi032.stdout:ceph: stdout ceph version 19.0.0-3450-g9a047fe7 (9a047fe7599a549042d06a3fb5b6813a203eabea) squid (dev) 2024-05-03T18:36:36.721 INFO:teuthology.orchestra.run.smithi032.stdout:Ceph version: ceph version 19.0.0-3450-g9a047fe7 (9a047fe7599a549042d06a3fb5b6813a203eabea) squid (dev) 2024-05-03T18:36:36.721 INFO:teuthology.orchestra.run.smithi032.stdout:Extracting ceph user uid/gid from container image... 2024-05-03T18:36:37.669 INFO:teuthology.orchestra.run.smithi032.stdout:stat: stdout 167 167 2024-05-03T18:36:37.669 INFO:teuthology.orchestra.run.smithi032.stdout:Creating initial keys... 2024-05-03T18:36:38.624 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph-authtool: stdout AQC2LjVmJTo0ABAAHVWXAQypmMU3azDyOr5d1g== 2024-05-03T18:36:39.538 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph-authtool: stdout AQC2LjVmymqrNhAAj27uPZ1t5woVlBzsDUVbIw== 2024-05-03T18:36:40.410 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph-authtool: stdout AQC3LjVmZegYNRAAz/MzyrLzdj6vYktkmBRm+A== 2024-05-03T18:36:40.411 INFO:teuthology.orchestra.run.smithi032.stdout:Creating initial monmap... 2024-05-03T18:36:41.290 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-05-03T18:36:41.291 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-05-03T18:36:41.291 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to f500455e-097b-11ef-bc97-c7b262605968 2024-05-03T18:36:41.291 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-05-03T18:36:41.291 INFO:teuthology.orchestra.run.smithi032.stdout:monmaptool for a [v2:172.21.15.32:3300,v1:172.21.15.32:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-05-03T18:36:41.291 INFO:teuthology.orchestra.run.smithi032.stdout:setting min_mon_release = quincy 2024-05-03T18:36:41.291 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/monmaptool: set fsid to f500455e-097b-11ef-bc97-c7b262605968 2024-05-03T18:36:41.291 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-05-03T18:36:41.291 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-03T18:36:41.291 INFO:teuthology.orchestra.run.smithi032.stdout:Creating mon... 2024-05-03T18:36:42.204 INFO:teuthology.orchestra.run.smithi032.stdout:create mon.a on 2024-05-03T18:36:42.424 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-05-03T18:36:42.587 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-05-03T18:36:42.855 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-f500455e-097b-11ef-bc97-c7b262605968.target → /etc/systemd/system/ceph-f500455e-097b-11ef-bc97-c7b262605968.target. 2024-05-03T18:36:42.855 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-f500455e-097b-11ef-bc97-c7b262605968.target → /etc/systemd/system/ceph-f500455e-097b-11ef-bc97-c7b262605968.target. 2024-05-03T18:36:43.092 INFO:teuthology.orchestra.run.smithi032.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-f500455e-097b-11ef-bc97-c7b262605968@mon.a 2024-05-03T18:36:43.092 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stderr Failed to reset failed state of unit ceph-f500455e-097b-11ef-bc97-c7b262605968@mon.a.service: Unit ceph-f500455e-097b-11ef-bc97-c7b262605968@mon.a.service not loaded. 2024-05-03T18:36:43.273 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-f500455e-097b-11ef-bc97-c7b262605968.target.wants/ceph-f500455e-097b-11ef-bc97-c7b262605968@mon.a.service → /etc/systemd/system/ceph-f500455e-097b-11ef-bc97-c7b262605968@.service. 2024-05-03T18:36:43.773 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:43 smithi032 ceph-mon[29390]: mon.a is new leader, mons a in quorum (ranks 0) 2024-05-03T18:36:43.787 INFO:teuthology.orchestra.run.smithi032.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-03T18:36:43.787 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stdout disabled 2024-05-03T18:36:43.794 INFO:teuthology.orchestra.run.smithi032.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-03T18:36:43.794 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stdout inactive 2024-05-03T18:36:43.794 INFO:teuthology.orchestra.run.smithi032.stdout:firewalld.service is not enabled 2024-05-03T18:36:43.794 INFO:teuthology.orchestra.run.smithi032.stdout:Not possible to enable service . firewalld.service is not available 2024-05-03T18:36:43.794 INFO:teuthology.orchestra.run.smithi032.stdout:Waiting for mon to start... 2024-05-03T18:36:43.795 INFO:teuthology.orchestra.run.smithi032.stdout:Waiting for mon... 2024-05-03T18:36:44.027 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:43 smithi032 bash[29371]: ec307666dac8f2b2c472bc056ef16c668e86ef8447f16be235a54fe21c166995 2024-05-03T18:36:44.027 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:43 smithi032 systemd[1]: Started Ceph mon.a for f500455e-097b-11ef-bc97-c7b262605968. 2024-05-03T18:36:44.912 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:44 smithi032 ceph-mon[29390]: mon.a is new leader, mons a in quorum (ranks 0) 2024-05-03T18:36:44.913 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:44 smithi032 ceph-mon[29390]: monmap epoch 1 2024-05-03T18:36:44.913 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:44 smithi032 ceph-mon[29390]: fsid f500455e-097b-11ef-bc97-c7b262605968 2024-05-03T18:36:44.913 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:44 smithi032 ceph-mon[29390]: last_changed 2024-05-03T18:36:40.734476+0000 2024-05-03T18:36:44.913 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:44 smithi032 ceph-mon[29390]: created 2024-05-03T18:36:40.734476+0000 2024-05-03T18:36:44.913 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:44 smithi032 ceph-mon[29390]: min_mon_release 19 (squid) 2024-05-03T18:36:44.913 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:44 smithi032 ceph-mon[29390]: election_strategy: 1 2024-05-03T18:36:44.913 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:44 smithi032 ceph-mon[29390]: 0: [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] mon.a 2024-05-03T18:36:44.913 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:44 smithi032 ceph-mon[29390]: fsmap 2024-05-03T18:36:44.913 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:44 smithi032 ceph-mon[29390]: osdmap e1: 0 total, 0 up, 0 in 2024-05-03T18:36:44.913 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:44 smithi032 ceph-mon[29390]: mgrmap e1: no daemons active 2024-05-03T18:36:44.913 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:44 smithi032 ceph-mon[29390]: from='client.? 172.21.15.32:0/2488520822' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-05-03T18:36:44.915 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout cluster: 2024-05-03T18:36:44.915 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout id: f500455e-097b-11ef-bc97-c7b262605968 2024-05-03T18:36:44.915 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-05-03T18:36:44.921 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout 2024-05-03T18:36:44.922 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout services: 2024-05-03T18:36:44.922 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.590183s) 2024-05-03T18:36:44.922 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-05-03T18:36:44.922 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-05-03T18:36:44.922 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout 2024-05-03T18:36:44.922 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout data: 2024-05-03T18:36:44.922 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-05-03T18:36:44.922 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-05-03T18:36:44.922 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-05-03T18:36:44.922 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout pgs: 2024-05-03T18:36:44.922 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout 2024-05-03T18:36:44.922 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:44.259+0000 7f7eebf23700 1 Processor -- start 2024-05-03T18:36:44.923 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:44.259+0000 7f7eebf23700 1 -- start start 2024-05-03T18:36:44.923 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:44.259+0000 7f7eebf23700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ee4109010 0x7f7ee4109430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:36:44.923 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:44.259+0000 7f7eebf23700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f7ee4109a00 con 0x7f7ee4109010 2024-05-03T18:36:44.923 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:44.259+0000 7f7ee9cbf700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ee4109010 0x7f7ee4109430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:36:44.923 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:44.259+0000 7f7ee9cbf700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ee4109010 0x7f7ee4109430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60776/0 (socket says 172.21.15.32:60776) 2024-05-03T18:36:44.923 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:44.259+0000 7f7ee9cbf700 1 -- 172.21.15.32:0/1422787583 learned_addr learned my addr 172.21.15.32:0/1422787583 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:36:44.923 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:44.260+0000 7f7ee9cbf700 1 -- 172.21.15.32:0/1422787583 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ee410a170 con 0x7f7ee4109010 2024-05-03T18:36:44.923 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:44.260+0000 7f7ee9cbf700 1 --2- 172.21.15.32:0/1422787583 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ee4109010 0x7f7ee4109430 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f7ee00098a0 tx=0x7f7ee0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2af6d6f43b59ade6 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:36:44.923 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:44.260+0000 7f7ee8cbd700 1 -- 172.21.15.32:0/1422787583 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7ee0004160 con 0x7f7ee4109010 2024-05-03T18:36:44.923 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:44.260+0000 7f7ee8cbd700 1 -- 172.21.15.32:0/1422787583 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ee0004300 con 0x7f7ee4109010 2024-05-03T18:36:44.923 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:44.260+0000 7f7ee8cbd700 1 -- 172.21.15.32:0/1422787583 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7ee00036a0 con 0x7f7ee4109010 2024-05-03T18:36:44.923 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:44.260+0000 7f7eebf23700 1 -- 172.21.15.32:0/1422787583 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ee4109010 msgr2=0x7f7ee4109430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:36:44.923 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:44.260+0000 7f7eebf23700 1 --2- 172.21.15.32:0/1422787583 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ee4109010 0x7f7ee4109430 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f7ee00098a0 tx=0x7f7ee0009bb0 comp rx=0 tx=0).stop 2024-05-03T18:36:44.923 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:44.261+0000 7f7eebf23700 1 -- 172.21.15.32:0/1422787583 shutdown_connections 2024-05-03T18:36:44.924 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:44.261+0000 7f7eebf23700 1 --2- 172.21.15.32:0/1422787583 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ee4109010 0x7f7ee4109430 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:44.924 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:44.261+0000 7f7eebf23700 1 -- 172.21.15.32:0/1422787583 >> 172.21.15.32:0/1422787583 conn(0x7f7ee4104650 msgr2=0x7f7ee4106a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:36:44.924 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:44.261+0000 7f7eebf23700 1 -- 172.21.15.32:0/1422787583 shutdown_connections 2024-05-03T18:36:44.924 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:44.261+0000 7f7eebf23700 1 -- 172.21.15.32:0/1422787583 wait complete. 2024-05-03T18:36:44.924 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:44.261+0000 7f7eebf23700 1 Processor -- start 2024-05-03T18:36:44.924 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:44.261+0000 7f7eebf23700 1 -- start start 2024-05-03T18:36:44.924 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:44.261+0000 7f7eebf23700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ee4109010 0x7f7ee407eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:36:44.924 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:44.261+0000 7f7eebf23700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f7ee410a6a0 con 0x7f7ee4109010 2024-05-03T18:36:44.924 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:44.262+0000 7f7ee9cbf700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ee4109010 0x7f7ee407eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:36:44.924 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:44.262+0000 7f7ee9cbf700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ee4109010 0x7f7ee407eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60780/0 (socket says 172.21.15.32:60780) 2024-05-03T18:36:44.924 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:44.262+0000 7f7ee9cbf700 1 -- 172.21.15.32:0/2488520822 learned_addr learned my addr 172.21.15.32:0/2488520822 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:36:44.924 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:44.262+0000 7f7ee9cbf700 1 -- 172.21.15.32:0/2488520822 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ee407f0c0 con 0x7f7ee4109010 2024-05-03T18:36:44.924 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:44.262+0000 7f7ee9cbf700 1 --2- 172.21.15.32:0/2488520822 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ee4109010 0x7f7ee407eb80 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f7ee0000c00 tx=0x7f7ee0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:36:44.925 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:44.262+0000 7f7edaffd700 1 -- 172.21.15.32:0/2488520822 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7ee0003db0 con 0x7f7ee4109010 2024-05-03T18:36:44.925 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:44.262+0000 7f7edaffd700 1 -- 172.21.15.32:0/2488520822 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ee0011630 con 0x7f7ee4109010 2024-05-03T18:36:44.925 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:44.262+0000 7f7eebf23700 1 -- 172.21.15.32:0/2488520822 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ee407d2b0 con 0x7f7ee4109010 2024-05-03T18:36:44.925 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:44.262+0000 7f7edaffd700 1 -- 172.21.15.32:0/2488520822 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7ee0011950 con 0x7f7ee4109010 2024-05-03T18:36:44.925 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:44.262+0000 7f7eebf23700 1 -- 172.21.15.32:0/2488520822 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ee407d7f0 con 0x7f7ee4109010 2024-05-03T18:36:44.925 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:44.262+0000 7f7edaffd700 1 -- 172.21.15.32:0/2488520822 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f7ee001a460 con 0x7f7ee4109010 2024-05-03T18:36:44.925 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:44.262+0000 7f7edaffd700 1 -- 172.21.15.32:0/2488520822 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f7ee001b780 con 0x7f7ee4109010 2024-05-03T18:36:44.925 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:44.263+0000 7f7eebf23700 1 -- 172.21.15.32:0/2488520822 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ee4064b00 con 0x7f7ee4109010 2024-05-03T18:36:44.925 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:44.266+0000 7f7edaffd700 1 -- 172.21.15.32:0/2488520822 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f7ee0031020 con 0x7f7ee4109010 2024-05-03T18:36:44.925 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:44.339+0000 7f7eebf23700 1 -- 172.21.15.32:0/2488520822 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f7ee4195fe0 con 0x7f7ee4109010 2024-05-03T18:36:44.925 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:44.339+0000 7f7edaffd700 1 -- 172.21.15.32:0/2488520822 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7f7ee001b9d0 con 0x7f7ee4109010 2024-05-03T18:36:44.925 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:44.340+0000 7f7eebf23700 1 -- 172.21.15.32:0/2488520822 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ee4109010 msgr2=0x7f7ee407eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:36:44.925 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:44.340+0000 7f7eebf23700 1 --2- 172.21.15.32:0/2488520822 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ee4109010 0x7f7ee407eb80 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f7ee0000c00 tx=0x7f7ee0004750 comp rx=0 tx=0).stop 2024-05-03T18:36:44.925 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:44.340+0000 7f7eebf23700 1 -- 172.21.15.32:0/2488520822 shutdown_connections 2024-05-03T18:36:44.925 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:44.340+0000 7f7eebf23700 1 --2- 172.21.15.32:0/2488520822 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ee4109010 0x7f7ee407eb80 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:44.926 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:44.340+0000 7f7eebf23700 1 -- 172.21.15.32:0/2488520822 >> 172.21.15.32:0/2488520822 conn(0x7f7ee4104650 msgr2=0x7f7ee4193320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:36:44.926 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:44.340+0000 7f7eebf23700 1 -- 172.21.15.32:0/2488520822 shutdown_connections 2024-05-03T18:36:44.926 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:44.340+0000 7f7eebf23700 1 -- 172.21.15.32:0/2488520822 wait complete. 2024-05-03T18:36:44.926 INFO:teuthology.orchestra.run.smithi032.stdout:mon is available 2024-05-03T18:36:44.926 INFO:teuthology.orchestra.run.smithi032.stdout:Assimilating anything we can from ceph.conf... 2024-05-03T18:36:45.968 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:45 smithi032 ceph-mon[29390]: from='client.? 172.21.15.32:0/870989813' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-05-03T18:36:45.968 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:45 smithi032 ceph-mon[29390]: from='client.? 172.21.15.32:0/870989813' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-05-03T18:36:45.978 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout 2024-05-03T18:36:45.978 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout [global] 2024-05-03T18:36:45.978 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout fsid = f500455e-097b-11ef-bc97-c7b262605968 2024-05-03T18:36:45.978 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-05-03T18:36:45.978 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.32:3300,v1:172.21.15.32:6789] 2024-05-03T18:36:45.978 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-05-03T18:36:45.979 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-05-03T18:36:45.979 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-05-03T18:36:45.979 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-05-03T18:36:45.979 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout 2024-05-03T18:36:45.979 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout [mgr] 2024-05-03T18:36:45.979 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-05-03T18:36:45.979 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout 2024-05-03T18:36:45.979 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout [osd] 2024-05-03T18:36:45.979 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout bdev_async_discard = True 2024-05-03T18:36:45.979 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-05-03T18:36:45.979 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-05-03T18:36:45.979 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-05-03T18:36:45.979 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-05-03T18:36:45.979 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-05-03T18:36:45.980 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:45.319+0000 7f843f3f3700 1 Processor -- start 2024-05-03T18:36:45.980 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:45.319+0000 7f843f3f3700 1 -- start start 2024-05-03T18:36:45.980 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:45.319+0000 7f843f3f3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8438107e90 0x7f84381082b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:36:45.980 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:45.319+0000 7f843f3f3700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f84381087f0 con 0x7f8438107e90 2024-05-03T18:36:45.980 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:45.320+0000 7f843d18f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8438107e90 0x7f84381082b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:36:45.980 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:45.320+0000 7f843d18f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8438107e90 0x7f84381082b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60784/0 (socket says 172.21.15.32:60784) 2024-05-03T18:36:45.980 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:45.320+0000 7f843d18f700 1 -- 172.21.15.32:0/3607431428 learned_addr learned my addr 172.21.15.32:0/3607431428 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:36:45.980 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:45.320+0000 7f843d18f700 1 -- 172.21.15.32:0/3607431428 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8438108ff0 con 0x7f8438107e90 2024-05-03T18:36:45.980 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:45.320+0000 7f843d18f700 1 --2- 172.21.15.32:0/3607431428 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8438107e90 0x7f84381082b0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f84280098a0 tx=0x7f8428009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b309a79fb193d771 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:36:45.980 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:45.320+0000 7f842ffff700 1 -- 172.21.15.32:0/3607431428 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8428004160 con 0x7f8438107e90 2024-05-03T18:36:45.980 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:45.320+0000 7f842ffff700 1 -- 172.21.15.32:0/3607431428 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8428004300 con 0x7f8438107e90 2024-05-03T18:36:45.980 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:45.320+0000 7f842ffff700 1 -- 172.21.15.32:0/3607431428 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f84280036a0 con 0x7f8438107e90 2024-05-03T18:36:45.980 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:45.321+0000 7f843f3f3700 1 -- 172.21.15.32:0/3607431428 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8438107e90 msgr2=0x7f84381082b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:36:45.981 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:45.321+0000 7f843f3f3700 1 --2- 172.21.15.32:0/3607431428 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8438107e90 0x7f84381082b0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f84280098a0 tx=0x7f8428009bb0 comp rx=0 tx=0).stop 2024-05-03T18:36:45.981 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:45.321+0000 7f843f3f3700 1 -- 172.21.15.32:0/3607431428 shutdown_connections 2024-05-03T18:36:45.981 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:45.321+0000 7f843f3f3700 1 --2- 172.21.15.32:0/3607431428 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8438107e90 0x7f84381082b0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:45.981 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:45.321+0000 7f843f3f3700 1 -- 172.21.15.32:0/3607431428 >> 172.21.15.32:0/3607431428 conn(0x7f84381034d0 msgr2=0x7f84381058f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:36:45.981 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:45.321+0000 7f843f3f3700 1 -- 172.21.15.32:0/3607431428 shutdown_connections 2024-05-03T18:36:45.981 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:45.321+0000 7f843f3f3700 1 -- 172.21.15.32:0/3607431428 wait complete. 2024-05-03T18:36:45.981 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:45.322+0000 7f843f3f3700 1 Processor -- start 2024-05-03T18:36:45.981 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:45.322+0000 7f843f3f3700 1 -- start start 2024-05-03T18:36:45.981 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:45.322+0000 7f843f3f3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8438107e90 0x7f84380769a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:36:45.981 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:45.322+0000 7f843f3f3700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f84381094c0 con 0x7f8438107e90 2024-05-03T18:36:45.981 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:45.322+0000 7f843d18f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8438107e90 0x7f84380769a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:36:45.981 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:45.322+0000 7f843d18f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8438107e90 0x7f84380769a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60798/0 (socket says 172.21.15.32:60798) 2024-05-03T18:36:45.981 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:45.322+0000 7f843d18f700 1 -- 172.21.15.32:0/870989813 learned_addr learned my addr 172.21.15.32:0/870989813 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:36:45.981 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:45.322+0000 7f843d18f700 1 -- 172.21.15.32:0/870989813 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f843819f340 con 0x7f8438107e90 2024-05-03T18:36:45.982 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:45.322+0000 7f843d18f700 1 --2- 172.21.15.32:0/870989813 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8438107e90 0x7f84380769a0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f8428000c00 tx=0x7f8428004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:36:45.982 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:45.322+0000 7f842e7fc700 1 -- 172.21.15.32:0/870989813 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8428003db0 con 0x7f8438107e90 2024-05-03T18:36:45.982 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:45.322+0000 7f842e7fc700 1 -- 172.21.15.32:0/870989813 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8428011630 con 0x7f8438107e90 2024-05-03T18:36:45.982 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:45.323+0000 7f843f3f3700 1 -- 172.21.15.32:0/870989813 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f843819f510 con 0x7f8438107e90 2024-05-03T18:36:45.982 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:45.323+0000 7f842e7fc700 1 -- 172.21.15.32:0/870989813 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8428011950 con 0x7f8438107e90 2024-05-03T18:36:45.982 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:45.323+0000 7f843f3f3700 1 -- 172.21.15.32:0/870989813 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f843819f930 con 0x7f8438107e90 2024-05-03T18:36:45.982 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:45.323+0000 7f842e7fc700 1 -- 172.21.15.32:0/870989813 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f8428011af0 con 0x7f8438107e90 2024-05-03T18:36:45.982 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:45.323+0000 7f842e7fc700 1 -- 172.21.15.32:0/870989813 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f842801c790 con 0x7f8438107e90 2024-05-03T18:36:45.982 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:45.324+0000 7f843f3f3700 1 -- 172.21.15.32:0/870989813 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8438064b00 con 0x7f8438107e90 2024-05-03T18:36:45.982 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:45.326+0000 7f842e7fc700 1 -- 172.21.15.32:0/870989813 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f8428004260 con 0x7f8438107e90 2024-05-03T18:36:45.982 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:45.397+0000 7f843f3f3700 1 -- 172.21.15.32:0/870989813 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f843802e850 con 0x7f8438107e90 2024-05-03T18:36:45.982 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:45.400+0000 7f842e7fc700 1 -- 172.21.15.32:0/870989813 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+492 (secure 0 0 0) 0x7f842801ca30 con 0x7f8438107e90 2024-05-03T18:36:45.982 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:45.400+0000 7f842e7fc700 1 -- 172.21.15.32:0/870989813 <== mon.0 v2:172.21.15.32:3300/0 8 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f842801cc10 con 0x7f8438107e90 2024-05-03T18:36:45.983 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:45.401+0000 7f843f3f3700 1 -- 172.21.15.32:0/870989813 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8438107e90 msgr2=0x7f84380769a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:36:45.983 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:45.401+0000 7f843f3f3700 1 --2- 172.21.15.32:0/870989813 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8438107e90 0x7f84380769a0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f8428000c00 tx=0x7f8428004750 comp rx=0 tx=0).stop 2024-05-03T18:36:45.983 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:45.401+0000 7f843f3f3700 1 -- 172.21.15.32:0/870989813 shutdown_connections 2024-05-03T18:36:45.983 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:45.401+0000 7f843f3f3700 1 --2- 172.21.15.32:0/870989813 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8438107e90 0x7f84380769a0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:45.983 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:45.401+0000 7f843f3f3700 1 -- 172.21.15.32:0/870989813 >> 172.21.15.32:0/870989813 conn(0x7f84381034d0 msgr2=0x7f84381920f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:36:45.983 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:45.401+0000 7f843f3f3700 1 -- 172.21.15.32:0/870989813 shutdown_connections 2024-05-03T18:36:45.983 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:45.401+0000 7f843f3f3700 1 -- 172.21.15.32:0/870989813 wait complete. 2024-05-03T18:36:45.983 INFO:teuthology.orchestra.run.smithi032.stdout:Generating new minimal ceph.conf... 2024-05-03T18:36:46.934 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:46 smithi032 ceph-mon[29390]: from='client.? 172.21.15.32:0/3054669207' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:36:46.939 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:46.357+0000 7f21db2aa700 1 Processor -- start 2024-05-03T18:36:46.939 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:46.358+0000 7f21db2aa700 1 -- start start 2024-05-03T18:36:46.939 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:46.358+0000 7f21db2aa700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f21d4109010 0x7f21d4109430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:36:46.939 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:46.358+0000 7f21db2aa700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f21d4109a00 con 0x7f21d4109010 2024-05-03T18:36:46.939 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:46.358+0000 7f21d9046700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f21d4109010 0x7f21d4109430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:36:46.939 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:46.358+0000 7f21d9046700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f21d4109010 0x7f21d4109430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60808/0 (socket says 172.21.15.32:60808) 2024-05-03T18:36:46.939 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:46.358+0000 7f21d9046700 1 -- 172.21.15.32:0/297957748 learned_addr learned my addr 172.21.15.32:0/297957748 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:36:46.940 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:46.358+0000 7f21d9046700 1 -- 172.21.15.32:0/297957748 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21d410a170 con 0x7f21d4109010 2024-05-03T18:36:46.940 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:46.359+0000 7f21d9046700 1 --2- 172.21.15.32:0/297957748 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f21d4109010 0x7f21d4109430 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f21cc009e80 tx=0x7f21cc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f38991fccb7c7614 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:36:46.940 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:46.359+0000 7f21d3fff700 1 -- 172.21.15.32:0/297957748 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f21cc017070 con 0x7f21d4109010 2024-05-03T18:36:46.940 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:46.359+0000 7f21d3fff700 1 -- 172.21.15.32:0/297957748 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f21cc00b8c0 con 0x7f21d4109010 2024-05-03T18:36:46.940 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:46.359+0000 7f21d3fff700 1 -- 172.21.15.32:0/297957748 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f21cc004490 con 0x7f21d4109010 2024-05-03T18:36:46.940 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:46.359+0000 7f21db2aa700 1 -- 172.21.15.32:0/297957748 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f21d4109010 msgr2=0x7f21d4109430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:36:46.940 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:46.359+0000 7f21db2aa700 1 --2- 172.21.15.32:0/297957748 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f21d4109010 0x7f21d4109430 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f21cc009e80 tx=0x7f21cc00b330 comp rx=0 tx=0).stop 2024-05-03T18:36:46.940 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:46.359+0000 7f21db2aa700 1 -- 172.21.15.32:0/297957748 shutdown_connections 2024-05-03T18:36:46.940 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:46.359+0000 7f21db2aa700 1 --2- 172.21.15.32:0/297957748 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f21d4109010 0x7f21d4109430 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:46.940 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:46.359+0000 7f21db2aa700 1 -- 172.21.15.32:0/297957748 >> 172.21.15.32:0/297957748 conn(0x7f21d4104650 msgr2=0x7f21d4106a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:36:46.940 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:46.360+0000 7f21db2aa700 1 -- 172.21.15.32:0/297957748 shutdown_connections 2024-05-03T18:36:46.940 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:46.360+0000 7f21db2aa700 1 -- 172.21.15.32:0/297957748 wait complete. 2024-05-03T18:36:46.940 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:46.360+0000 7f21db2aa700 1 Processor -- start 2024-05-03T18:36:46.940 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:46.360+0000 7f21db2aa700 1 -- start start 2024-05-03T18:36:46.941 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:46.360+0000 7f21db2aa700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f21d4109010 0x7f21d419cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:36:46.941 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:46.360+0000 7f21db2aa700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f21d410d520 con 0x7f21d4109010 2024-05-03T18:36:46.941 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:46.361+0000 7f21d9046700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f21d4109010 0x7f21d419cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:36:46.941 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:46.361+0000 7f21d9046700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f21d4109010 0x7f21d419cee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60824/0 (socket says 172.21.15.32:60824) 2024-05-03T18:36:46.941 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:46.361+0000 7f21d9046700 1 -- 172.21.15.32:0/3054669207 learned_addr learned my addr 172.21.15.32:0/3054669207 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:36:46.941 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:46.361+0000 7f21d9046700 1 -- 172.21.15.32:0/3054669207 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21d419d420 con 0x7f21d4109010 2024-05-03T18:36:46.941 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:46.361+0000 7f21d9046700 1 --2- 172.21.15.32:0/3054669207 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f21d4109010 0x7f21d419cee0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f21cc004a60 tx=0x7f21cc00bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:36:46.941 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:46.361+0000 7f21d27fc700 1 -- 172.21.15.32:0/3054669207 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f21cc020020 con 0x7f21d4109010 2024-05-03T18:36:46.941 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:46.361+0000 7f21d27fc700 1 -- 172.21.15.32:0/3054669207 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f21cc012430 con 0x7f21d4109010 2024-05-03T18:36:46.941 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:46.361+0000 7f21db2aa700 1 -- 172.21.15.32:0/3054669207 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21d419d6b0 con 0x7f21d4109010 2024-05-03T18:36:46.941 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:46.361+0000 7f21d27fc700 1 -- 172.21.15.32:0/3054669207 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f21cc017040 con 0x7f21d4109010 2024-05-03T18:36:46.941 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:46.361+0000 7f21db2aa700 1 -- 172.21.15.32:0/3054669207 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21d41a03a0 con 0x7f21d4109010 2024-05-03T18:36:46.941 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:46.361+0000 7f21d27fc700 1 -- 172.21.15.32:0/3054669207 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f21cc023020 con 0x7f21d4109010 2024-05-03T18:36:46.941 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:46.361+0000 7f21d27fc700 1 -- 172.21.15.32:0/3054669207 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f21cc00f050 con 0x7f21d4109010 2024-05-03T18:36:46.941 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:46.362+0000 7f21db2aa700 1 -- 172.21.15.32:0/3054669207 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21d4064b00 con 0x7f21d4109010 2024-05-03T18:36:46.942 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:46.364+0000 7f21d27fc700 1 -- 172.21.15.32:0/3054669207 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f21cc02a020 con 0x7f21d4109010 2024-05-03T18:36:46.942 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:46.436+0000 7f21db2aa700 1 -- 172.21.15.32:0/3054669207 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f21d41a0570 con 0x7f21d4109010 2024-05-03T18:36:46.942 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:46.436+0000 7f21d27fc700 1 -- 172.21.15.32:0/3054669207 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7f21cc00b860 con 0x7f21d4109010 2024-05-03T18:36:46.942 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:46.437+0000 7f21db2aa700 1 -- 172.21.15.32:0/3054669207 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f21d4109010 msgr2=0x7f21d419cee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:36:46.942 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:46.437+0000 7f21db2aa700 1 --2- 172.21.15.32:0/3054669207 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f21d4109010 0x7f21d419cee0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f21cc004a60 tx=0x7f21cc00bd50 comp rx=0 tx=0).stop 2024-05-03T18:36:46.942 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:46.437+0000 7f21db2aa700 1 -- 172.21.15.32:0/3054669207 shutdown_connections 2024-05-03T18:36:46.942 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:46.437+0000 7f21db2aa700 1 --2- 172.21.15.32:0/3054669207 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f21d4109010 0x7f21d419cee0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:46.942 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:46.437+0000 7f21db2aa700 1 -- 172.21.15.32:0/3054669207 >> 172.21.15.32:0/3054669207 conn(0x7f21d4104650 msgr2=0x7f21d4105140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:36:46.942 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:46.437+0000 7f21db2aa700 1 -- 172.21.15.32:0/3054669207 shutdown_connections 2024-05-03T18:36:46.942 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:46.437+0000 7f21db2aa700 1 -- 172.21.15.32:0/3054669207 wait complete. 2024-05-03T18:36:46.942 INFO:teuthology.orchestra.run.smithi032.stdout:Restarting the monitor... 2024-05-03T18:36:47.228 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:46 smithi032 systemd[1]: Stopping Ceph mon.a for f500455e-097b-11ef-bc97-c7b262605968... 2024-05-03T18:36:47.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:47 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-mon-a[29385]: 2024-05-03T18:36:47.227+0000 7fb10e662700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-03T18:36:47.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:47 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-mon-a[29385]: 2024-05-03T18:36:47.227+0000 7fb10e662700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-05-03T18:36:47.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:47 smithi032 podman[29635]: 2024-05-03 18:36:47.443359576 +0000 UTC m=+0.324800878 container died ec307666dac8f2b2c472bc056ef16c668e86ef8447f16be235a54fe21c166995 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-mon-a, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240430) 2024-05-03T18:36:48.061 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:47 smithi032 podman[29635]: 2024-05-03 18:36:47.793738238 +0000 UTC m=+0.675179548 container cleanup ec307666dac8f2b2c472bc056ef16c668e86ef8447f16be235a54fe21c166995 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-mon-a, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240430, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe) 2024-05-03T18:36:48.061 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:47 smithi032 bash[29635]: ceph-f500455e-097b-11ef-bc97-c7b262605968-mon-a 2024-05-03T18:36:48.317 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 podman[29649]: 2024-05-03 18:36:48.061214095 +0000 UTC m=+0.616974128 container remove ec307666dac8f2b2c472bc056ef16c668e86ef8447f16be235a54fe21c166995 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-mon-a, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240430, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-05-03T18:36:48.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 systemd[1]: ceph-f500455e-097b-11ef-bc97-c7b262605968@mon.a.service: Deactivated successfully. 2024-05-03T18:36:48.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 systemd[1]: Stopped Ceph mon.a for f500455e-097b-11ef-bc97-c7b262605968. 2024-05-03T18:36:48.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 systemd[1]: Starting Ceph mon.a for f500455e-097b-11ef-bc97-c7b262605968... 2024-05-03T18:36:48.664 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 podman[29744]: 2024-05-03 18:36:48.40392262 +0000 UTC m=+0.109335075 container create b018994d5361d5479c162f8af535e5ecc6199a8c32a7196b0b9a5c86d802daf6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-mon-a, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240430, ceph=True) 2024-05-03T18:36:48.664 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 podman[29744]: 2024-05-03 18:36:48.312785225 +0000 UTC m=+0.018197690 image pull ee4d846202496ba6096593e8e8becc0700a5eb7ebd95536031187f5f337d4012 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea 2024-05-03T18:36:48.664 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 podman[29744]: 2024-05-03 18:36:48.614283968 +0000 UTC m=+0.319696424 container init b018994d5361d5479c162f8af535e5ecc6199a8c32a7196b0b9a5c86d802daf6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-mon-a, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240430, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-05-03T18:36:48.664 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 podman[29744]: 2024-05-03 18:36:48.617235726 +0000 UTC m=+0.322648191 container start b018994d5361d5479c162f8af535e5ecc6199a8c32a7196b0b9a5c86d802daf6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240430, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-05-03T18:36:48.664 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: set uid:gid to 167:167 (ceph:ceph) 2024-05-03T18:36:48.664 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: ceph version 19.0.0-3450-g9a047fe7 (9a047fe7599a549042d06a3fb5b6813a203eabea) squid (dev), process ceph-mon, pid 2 2024-05-03T18:36:48.665 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: pidfile_write: ignore empty --pid-file 2024-05-03T18:36:48.665 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: load: jerasure load: lrc 2024-05-03T18:36:48.665 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: RocksDB version: 7.9.2 2024-05-03T18:36:48.665 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Git sha 0 2024-05-03T18:36:48.665 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Compile date 2024-05-01 21:12:03 2024-05-03T18:36:48.665 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: DB SUMMARY 2024-05-03T18:36:48.665 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: DB Session ID: AM99ANTCF7JVLS8RAUD1 2024-05-03T18:36:48.665 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: CURRENT file: CURRENT 2024-05-03T18:36:48.665 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: IDENTITY file: IDENTITY 2024-05-03T18:36:48.665 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-05-03T18:36:48.665 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-05-03T18:36:48.665 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 105554 ; 2024-05-03T18:36:48.667 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.error_if_exists: 0 2024-05-03T18:36:48.667 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.create_if_missing: 0 2024-05-03T18:36:48.667 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.paranoid_checks: 1 2024-05-03T18:36:48.667 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.flush_verify_memtable_count: 1 2024-05-03T18:36:48.667 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-05-03T18:36:48.667 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-05-03T18:36:48.667 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.env: 0x5639745bacc0 2024-05-03T18:36:48.667 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.fs: PosixFileSystem 2024-05-03T18:36:48.667 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.info_log: 0x563975c2bda0 2024-05-03T18:36:48.667 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.max_file_opening_threads: 16 2024-05-03T18:36:48.667 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.statistics: (nil) 2024-05-03T18:36:48.668 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.use_fsync: 0 2024-05-03T18:36:48.668 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.max_log_file_size: 0 2024-05-03T18:36:48.668 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-05-03T18:36:48.668 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.log_file_time_to_roll: 0 2024-05-03T18:36:48.668 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.keep_log_file_num: 1000 2024-05-03T18:36:48.668 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.recycle_log_file_num: 0 2024-05-03T18:36:48.668 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.allow_fallocate: 1 2024-05-03T18:36:48.668 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.allow_mmap_reads: 0 2024-05-03T18:36:48.668 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.allow_mmap_writes: 0 2024-05-03T18:36:48.668 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.use_direct_reads: 0 2024-05-03T18:36:48.668 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-05-03T18:36:48.668 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.create_missing_column_families: 0 2024-05-03T18:36:48.668 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.db_log_dir: 2024-05-03T18:36:48.668 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.wal_dir: 2024-05-03T18:36:48.669 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.table_cache_numshardbits: 6 2024-05-03T18:36:48.669 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.WAL_ttl_seconds: 0 2024-05-03T18:36:48.669 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.WAL_size_limit_MB: 0 2024-05-03T18:36:48.669 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-05-03T18:36:48.669 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-05-03T18:36:48.669 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.is_fd_close_on_exec: 1 2024-05-03T18:36:48.669 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.advise_random_on_open: 1 2024-05-03T18:36:48.669 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.db_write_buffer_size: 0 2024-05-03T18:36:48.669 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.write_buffer_manager: 0x563975bfd0e0 2024-05-03T18:36:48.669 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-05-03T18:36:48.669 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-05-03T18:36:48.669 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.use_adaptive_mutex: 0 2024-05-03T18:36:48.669 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.rate_limiter: (nil) 2024-05-03T18:36:48.669 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-05-03T18:36:48.669 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.wal_recovery_mode: 2 2024-05-03T18:36:48.670 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.enable_thread_tracking: 0 2024-05-03T18:36:48.670 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.enable_pipelined_write: 0 2024-05-03T18:36:48.670 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.unordered_write: 0 2024-05-03T18:36:48.670 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-05-03T18:36:48.670 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-05-03T18:36:48.670 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-05-03T18:36:48.670 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-05-03T18:36:48.670 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.row_cache: None 2024-05-03T18:36:48.670 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.wal_filter: None 2024-05-03T18:36:48.670 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-05-03T18:36:48.670 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.allow_ingest_behind: 0 2024-05-03T18:36:48.670 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.two_write_queues: 0 2024-05-03T18:36:48.670 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.manual_wal_flush: 0 2024-05-03T18:36:48.670 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.wal_compression: 0 2024-05-03T18:36:48.671 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.atomic_flush: 0 2024-05-03T18:36:48.671 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-05-03T18:36:48.671 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.persist_stats_to_disk: 0 2024-05-03T18:36:48.671 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.write_dbid_to_manifest: 0 2024-05-03T18:36:48.671 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.log_readahead_size: 0 2024-05-03T18:36:48.671 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-05-03T18:36:48.671 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.best_efforts_recovery: 0 2024-05-03T18:36:48.671 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-05-03T18:36:48.671 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-05-03T18:36:48.671 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.allow_data_in_errors: 0 2024-05-03T18:36:48.671 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.db_host_id: __hostname__ 2024-05-03T18:36:48.671 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.enforce_single_del_contracts: true 2024-05-03T18:36:48.671 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.max_background_jobs: 2 2024-05-03T18:36:48.671 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.max_background_compactions: -1 2024-05-03T18:36:48.672 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.max_subcompactions: 1 2024-05-03T18:36:48.672 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-05-03T18:36:48.672 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-05-03T18:36:48.672 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.delayed_write_rate : 16777216 2024-05-03T18:36:48.672 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.max_total_wal_size: 0 2024-05-03T18:36:48.672 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-05-03T18:36:48.672 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.stats_dump_period_sec: 600 2024-05-03T18:36:48.672 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.stats_persist_period_sec: 600 2024-05-03T18:36:48.672 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-05-03T18:36:48.672 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.max_open_files: -1 2024-05-03T18:36:48.672 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.bytes_per_sync: 0 2024-05-03T18:36:48.672 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.wal_bytes_per_sync: 0 2024-05-03T18:36:48.672 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.strict_bytes_per_sync: 0 2024-05-03T18:36:48.672 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.compaction_readahead_size: 0 2024-05-03T18:36:48.673 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.max_background_flushes: -1 2024-05-03T18:36:48.673 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Compression algorithms supported: 2024-05-03T18:36:48.673 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-05-03T18:36:48.673 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: kZSTD supported: 0 2024-05-03T18:36:48.673 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: kXpressCompression supported: 0 2024-05-03T18:36:48.673 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: kLZ4HCCompression supported: 1 2024-05-03T18:36:48.673 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: kZlibCompression supported: 1 2024-05-03T18:36:48.673 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: kSnappyCompression supported: 1 2024-05-03T18:36:48.673 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: kLZ4Compression supported: 1 2024-05-03T18:36:48.673 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: kBZip2Compression supported: 0 2024-05-03T18:36:48.673 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Fast CRC32 supported: Supported on x86 2024-05-03T18:36:48.673 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: DMutex implementation: pthread_mutex_t 2024-05-03T18:36:48.673 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-05-03T18:36:48.673 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-05-03T18:36:48.673 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-05-03T18:36:48.674 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.merge_operator: 2024-05-03T18:36:48.674 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.compaction_filter: None 2024-05-03T18:36:48.674 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.compaction_filter_factory: None 2024-05-03T18:36:48.674 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.sst_partitioner_factory: None 2024-05-03T18:36:48.674 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.memtable_factory: SkipListFactory 2024-05-03T18:36:48.674 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.table_factory: BlockBasedTable 2024-05-03T18:36:48.674 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x563975c2bf60) 2024-05-03T18:36:48.674 INFO:journalctl@ceph.mon.a.smithi032.stdout: cache_index_and_filter_blocks: 1 2024-05-03T18:36:48.674 INFO:journalctl@ceph.mon.a.smithi032.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-05-03T18:36:48.674 INFO:journalctl@ceph.mon.a.smithi032.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-05-03T18:36:48.674 INFO:journalctl@ceph.mon.a.smithi032.stdout: pin_top_level_index_and_filter: 1 2024-05-03T18:36:48.674 INFO:journalctl@ceph.mon.a.smithi032.stdout: index_type: 0 2024-05-03T18:36:48.674 INFO:journalctl@ceph.mon.a.smithi032.stdout: data_block_index_type: 0 2024-05-03T18:36:48.674 INFO:journalctl@ceph.mon.a.smithi032.stdout: index_shortening: 1 2024-05-03T18:36:48.675 INFO:journalctl@ceph.mon.a.smithi032.stdout: data_block_hash_table_util_ratio: 0.750000 2024-05-03T18:36:48.675 INFO:journalctl@ceph.mon.a.smithi032.stdout: checksum: 4 2024-05-03T18:36:48.675 INFO:journalctl@ceph.mon.a.smithi032.stdout: no_block_cache: 0 2024-05-03T18:36:48.675 INFO:journalctl@ceph.mon.a.smithi032.stdout: block_cache: 0x563975c2f1f0 2024-05-03T18:36:48.675 INFO:journalctl@ceph.mon.a.smithi032.stdout: block_cache_name: BinnedLRUCache 2024-05-03T18:36:48.675 INFO:journalctl@ceph.mon.a.smithi032.stdout: block_cache_options: 2024-05-03T18:36:48.675 INFO:journalctl@ceph.mon.a.smithi032.stdout: capacity : 536870912 2024-05-03T18:36:48.675 INFO:journalctl@ceph.mon.a.smithi032.stdout: num_shard_bits : 4 2024-05-03T18:36:48.675 INFO:journalctl@ceph.mon.a.smithi032.stdout: strict_capacity_limit : 0 2024-05-03T18:36:48.675 INFO:journalctl@ceph.mon.a.smithi032.stdout: high_pri_pool_ratio: 0.000 2024-05-03T18:36:48.675 INFO:journalctl@ceph.mon.a.smithi032.stdout: block_cache_compressed: (nil) 2024-05-03T18:36:48.675 INFO:journalctl@ceph.mon.a.smithi032.stdout: persistent_cache: (nil) 2024-05-03T18:36:48.675 INFO:journalctl@ceph.mon.a.smithi032.stdout: block_size: 4096 2024-05-03T18:36:48.675 INFO:journalctl@ceph.mon.a.smithi032.stdout: block_size_deviation: 10 2024-05-03T18:36:48.675 INFO:journalctl@ceph.mon.a.smithi032.stdout: block_restart_interval: 16 2024-05-03T18:36:48.676 INFO:journalctl@ceph.mon.a.smithi032.stdout: index_block_restart_interval: 1 2024-05-03T18:36:48.676 INFO:journalctl@ceph.mon.a.smithi032.stdout: metadata_block_size: 4096 2024-05-03T18:36:48.676 INFO:journalctl@ceph.mon.a.smithi032.stdout: partition_filters: 0 2024-05-03T18:36:48.676 INFO:journalctl@ceph.mon.a.smithi032.stdout: use_delta_encoding: 1 2024-05-03T18:36:48.676 INFO:journalctl@ceph.mon.a.smithi032.stdout: filter_policy: bloomfilter 2024-05-03T18:36:48.676 INFO:journalctl@ceph.mon.a.smithi032.stdout: whole_key_filtering: 1 2024-05-03T18:36:48.676 INFO:journalctl@ceph.mon.a.smithi032.stdout: verify_compression: 0 2024-05-03T18:36:48.676 INFO:journalctl@ceph.mon.a.smithi032.stdout: read_amp_bytes_per_bit: 0 2024-05-03T18:36:48.676 INFO:journalctl@ceph.mon.a.smithi032.stdout: format_version: 5 2024-05-03T18:36:48.676 INFO:journalctl@ceph.mon.a.smithi032.stdout: enable_index_compression: 1 2024-05-03T18:36:48.676 INFO:journalctl@ceph.mon.a.smithi032.stdout: block_align: 0 2024-05-03T18:36:48.676 INFO:journalctl@ceph.mon.a.smithi032.stdout: max_auto_readahead_size: 262144 2024-05-03T18:36:48.676 INFO:journalctl@ceph.mon.a.smithi032.stdout: prepopulate_block_cache: 0 2024-05-03T18:36:48.676 INFO:journalctl@ceph.mon.a.smithi032.stdout: initial_auto_readahead_size: 8192 2024-05-03T18:36:48.676 INFO:journalctl@ceph.mon.a.smithi032.stdout: num_file_reads_for_auto_readahead: 2 2024-05-03T18:36:48.677 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.write_buffer_size: 33554432 2024-05-03T18:36:48.677 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.max_write_buffer_number: 2 2024-05-03T18:36:48.677 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.compression: NoCompression 2024-05-03T18:36:48.677 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.bottommost_compression: Disabled 2024-05-03T18:36:48.677 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.prefix_extractor: nullptr 2024-05-03T18:36:48.677 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-05-03T18:36:48.677 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.num_levels: 7 2024-05-03T18:36:48.677 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-05-03T18:36:48.677 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-05-03T18:36:48.677 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-05-03T18:36:48.677 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-05-03T18:36:48.677 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-05-03T18:36:48.677 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-05-03T18:36:48.677 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-05-03T18:36:48.678 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-05-03T18:36:48.678 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-05-03T18:36:48.678 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-05-03T18:36:48.678 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-05-03T18:36:48.678 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-05-03T18:36:48.678 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.compression_opts.window_bits: -14 2024-05-03T18:36:48.678 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.compression_opts.level: 32767 2024-05-03T18:36:48.678 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.compression_opts.strategy: 0 2024-05-03T18:36:48.678 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-05-03T18:36:48.678 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-05-03T18:36:48.678 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-05-03T18:36:48.678 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-05-03T18:36:48.678 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.compression_opts.enabled: false 2024-05-03T18:36:48.678 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-05-03T18:36:48.678 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-05-03T18:36:48.679 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-05-03T18:36:48.679 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-05-03T18:36:48.679 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.target_file_size_base: 67108864 2024-05-03T18:36:48.679 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.target_file_size_multiplier: 1 2024-05-03T18:36:48.679 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-05-03T18:36:48.679 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-05-03T18:36:48.679 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-05-03T18:36:48.679 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-05-03T18:36:48.679 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-05-03T18:36:48.679 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-05-03T18:36:48.679 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-05-03T18:36:48.679 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-05-03T18:36:48.679 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-05-03T18:36:48.679 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-05-03T18:36:48.679 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-05-03T18:36:48.680 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-05-03T18:36:48.680 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-05-03T18:36:48.680 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.arena_block_size: 1048576 2024-05-03T18:36:48.680 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-05-03T18:36:48.680 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-05-03T18:36:48.680 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.disable_auto_compactions: 0 2024-05-03T18:36:48.680 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-05-03T18:36:48.680 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-05-03T18:36:48.680 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-05-03T18:36:48.680 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-05-03T18:36:48.680 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-05-03T18:36:48.680 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-05-03T18:36:48.680 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-05-03T18:36:48.680 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-05-03T18:36:48.681 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-05-03T18:36:48.681 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-05-03T18:36:48.681 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-05-03T18:36:48.681 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.inplace_update_support: 0 2024-05-03T18:36:48.681 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.inplace_update_num_locks: 10000 2024-05-03T18:36:48.681 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-05-03T18:36:48.681 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-05-03T18:36:48.681 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.memtable_huge_page_size: 0 2024-05-03T18:36:48.681 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.bloom_locality: 0 2024-05-03T18:36:48.681 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.max_successive_merges: 0 2024-05-03T18:36:48.681 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.optimize_filters_for_hits: 0 2024-05-03T18:36:48.681 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.paranoid_file_checks: 0 2024-05-03T18:36:48.681 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.force_consistency_checks: 1 2024-05-03T18:36:48.681 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.report_bg_io_stats: 0 2024-05-03T18:36:48.681 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.ttl: 2592000 2024-05-03T18:36:48.682 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.periodic_compaction_seconds: 0 2024-05-03T18:36:48.682 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-05-03T18:36:48.682 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-05-03T18:36:48.682 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.enable_blob_files: false 2024-05-03T18:36:48.682 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.min_blob_size: 0 2024-05-03T18:36:48.682 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.blob_file_size: 268435456 2024-05-03T18:36:48.682 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.blob_compression_type: NoCompression 2024-05-03T18:36:48.682 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.enable_blob_garbage_collection: false 2024-05-03T18:36:48.682 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-05-03T18:36:48.682 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-05-03T18:36:48.682 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-05-03T18:36:48.682 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.blob_file_starting_level: 0 2024-05-03T18:36:48.682 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-05-03T18:36:48.682 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-05-03T18:36:48.683 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-05-03T18:36:48.683 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: b21fc0b5-ea6c-4a0e-81af-fdfb48e7d0ef 2024-05-03T18:36:48.683 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714761408646937, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-05-03T18:36:48.683 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-05-03T18:36:48.683 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714761408647965, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 100659, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 328, "table_properties": {"data_size": 98547, "index_size": 304, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14294, "raw_average_key_size": 49, "raw_value_size": 90775, "raw_average_value_size": 316, "num_data_blocks": 12, "num_entries": 287, "num_filter_entries": 287, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1714761408, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "b21fc0b5-ea6c-4a0e-81af-fdfb48e7d0ef", "db_session_id": "AM99ANTCF7JVLS8RAUD1", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-05-03T18:36:48.683 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714761408648051, "job": 1, "event": "recovery_finished"} 2024-05-03T18:36:48.683 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-05-03T18:36:48.683 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-03T18:36:48.683 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x563975d4c000 2024-05-03T18:36:48.683 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: DB pointer 0x563975d38000 2024-05-03T18:36:48.683 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-05-03T18:36:48.683 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-05-03T18:36:48.683 INFO:journalctl@ceph.mon.a.smithi032.stdout: ** DB Stats ** 2024-05-03T18:36:48.684 INFO:journalctl@ceph.mon.a.smithi032.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-03T18:36:48.684 INFO:journalctl@ceph.mon.a.smithi032.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-03T18:36:48.684 INFO:journalctl@ceph.mon.a.smithi032.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-03T18:36:48.684 INFO:journalctl@ceph.mon.a.smithi032.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-03T18:36:48.684 INFO:journalctl@ceph.mon.a.smithi032.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-03T18:36:48.684 INFO:journalctl@ceph.mon.a.smithi032.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-03T18:36:48.684 INFO:journalctl@ceph.mon.a.smithi032.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-03T18:36:48.684 INFO:journalctl@ceph.mon.a.smithi032.stdout: 2024-05-03T18:36:48.684 INFO:journalctl@ceph.mon.a.smithi032.stdout: ** Compaction Stats [default] ** 2024-05-03T18:36:48.684 INFO:journalctl@ceph.mon.a.smithi032.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-03T18:36:48.684 INFO:journalctl@ceph.mon.a.smithi032.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-05-03T18:36:48.684 INFO:journalctl@ceph.mon.a.smithi032.stdout: L0 2/0 100.16 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 157.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-03T18:36:48.684 INFO:journalctl@ceph.mon.a.smithi032.stdout: Sum 2/0 100.16 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 157.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-03T18:36:48.684 INFO:journalctl@ceph.mon.a.smithi032.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 157.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-03T18:36:48.684 INFO:journalctl@ceph.mon.a.smithi032.stdout: 2024-05-03T18:36:48.685 INFO:journalctl@ceph.mon.a.smithi032.stdout: ** Compaction Stats [default] ** 2024-05-03T18:36:48.685 INFO:journalctl@ceph.mon.a.smithi032.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-03T18:36:48.685 INFO:journalctl@ceph.mon.a.smithi032.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-03T18:36:48.685 INFO:journalctl@ceph.mon.a.smithi032.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 157.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-03T18:36:48.685 INFO:journalctl@ceph.mon.a.smithi032.stdout: 2024-05-03T18:36:48.685 INFO:journalctl@ceph.mon.a.smithi032.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-05-03T18:36:48.685 INFO:journalctl@ceph.mon.a.smithi032.stdout: 2024-05-03T18:36:48.685 INFO:journalctl@ceph.mon.a.smithi032.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-03T18:36:48.685 INFO:journalctl@ceph.mon.a.smithi032.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-05-03T18:36:48.685 INFO:journalctl@ceph.mon.a.smithi032.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-03T18:36:48.685 INFO:journalctl@ceph.mon.a.smithi032.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-03T18:36:48.685 INFO:journalctl@ceph.mon.a.smithi032.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-03T18:36:48.685 INFO:journalctl@ceph.mon.a.smithi032.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-03T18:36:48.685 INFO:journalctl@ceph.mon.a.smithi032.stdout: Cumulative compaction: 0.00 GB write, 31.38 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-03T18:36:48.686 INFO:journalctl@ceph.mon.a.smithi032.stdout: Interval compaction: 0.00 GB write, 31.38 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-03T18:36:48.686 INFO:journalctl@ceph.mon.a.smithi032.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-03T18:36:48.686 INFO:journalctl@ceph.mon.a.smithi032.stdout: Block cache BinnedLRUCache@0x563975c2f1f0#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8e-06 secs_since: 0 2024-05-03T18:36:48.686 INFO:journalctl@ceph.mon.a.smithi032.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-05-03T18:36:48.686 INFO:journalctl@ceph.mon.a.smithi032.stdout: 2024-05-03T18:36:48.686 INFO:journalctl@ceph.mon.a.smithi032.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-03T18:36:48.686 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: starting mon.a rank 0 at public addrs [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] at bind addrs [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid f500455e-097b-11ef-bc97-c7b262605968 2024-05-03T18:36:48.686 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: mon.a@-1(???) e1 preinit fsid f500455e-097b-11ef-bc97-c7b262605968 2024-05-03T18:36:48.686 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: mon.a@-1(???).mds e1 new map 2024-05-03T18:36:48.686 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: mon.a@-1(???).mds e1 print_map 2024-05-03T18:36:48.686 INFO:journalctl@ceph.mon.a.smithi032.stdout: e1 2024-05-03T18:36:48.686 INFO:journalctl@ceph.mon.a.smithi032.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-03T18:36:48.686 INFO:journalctl@ceph.mon.a.smithi032.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-03T18:36:48.687 INFO:journalctl@ceph.mon.a.smithi032.stdout: legacy client fscid: -1 2024-05-03T18:36:48.687 INFO:journalctl@ceph.mon.a.smithi032.stdout: 2024-05-03T18:36:48.687 INFO:journalctl@ceph.mon.a.smithi032.stdout: No filesystems configured 2024-05-03T18:36:48.687 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-05-03T18:36:48.687 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-03T18:36:48.687 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-03T18:36:48.687 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-03T18:36:48.687 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-05-03T18:36:48.687 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 bash[29744]: b018994d5361d5479c162f8af535e5ecc6199a8c32a7196b0b9a5c86d802daf6 2024-05-03T18:36:48.687 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 systemd[1]: Started Ceph mon.a for f500455e-097b-11ef-bc97-c7b262605968. 2024-05-03T18:36:48.688 INFO:teuthology.orchestra.run.smithi032.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-05-03T18:36:48.934 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: mon.a is new leader, mons a in quorum (ranks 0) 2024-05-03T18:36:48.934 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: monmap epoch 1 2024-05-03T18:36:48.934 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: fsid f500455e-097b-11ef-bc97-c7b262605968 2024-05-03T18:36:48.934 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: last_changed 2024-05-03T18:36:40.734476+0000 2024-05-03T18:36:48.934 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: created 2024-05-03T18:36:40.734476+0000 2024-05-03T18:36:48.934 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: min_mon_release 19 (squid) 2024-05-03T18:36:48.934 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: election_strategy: 1 2024-05-03T18:36:48.935 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: 0: [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] mon.a 2024-05-03T18:36:48.935 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: fsmap 2024-05-03T18:36:48.935 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: osdmap e1: 0 total, 0 up, 0 in 2024-05-03T18:36:48.935 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:48 smithi032 ceph-mon[29763]: mgrmap e1: no daemons active 2024-05-03T18:36:49.827 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:49.139+0000 7f9ffd918700 1 Processor -- start 2024-05-03T18:36:49.827 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:49.139+0000 7f9ffd918700 1 -- start start 2024-05-03T18:36:49.827 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:49.139+0000 7f9ffd918700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ff8109a50 0x7f9ff810be40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:36:49.827 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:49.139+0000 7f9ffd918700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f9ff807cec0 con 0x7f9ff8109a50 2024-05-03T18:36:49.828 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:49.139+0000 7f9ff6ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ff8109a50 0x7f9ff810be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:36:49.828 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:49.139+0000 7f9ff6ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ff8109a50 0x7f9ff810be40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60840/0 (socket says 172.21.15.32:60840) 2024-05-03T18:36:49.828 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:49.139+0000 7f9ff6ffd700 1 -- 172.21.15.32:0/3109296763 learned_addr learned my addr 172.21.15.32:0/3109296763 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:36:49.828 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:49.140+0000 7f9ff6ffd700 1 -- 172.21.15.32:0/3109296763 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ff810c380 con 0x7f9ff8109a50 2024-05-03T18:36:49.828 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:49.140+0000 7f9ff6ffd700 1 --2- 172.21.15.32:0/3109296763 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ff8109a50 0x7f9ff810be40 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f9fe00098a0 tx=0x7f9fe0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4ca65a8548962d8 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:36:49.828 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:49.140+0000 7f9ff5ffb700 1 -- 172.21.15.32:0/3109296763 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9fe0004160 con 0x7f9ff8109a50 2024-05-03T18:36:49.828 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:49.140+0000 7f9ff5ffb700 1 -- 172.21.15.32:0/3109296763 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f9fe0004300 con 0x7f9ff8109a50 2024-05-03T18:36:49.828 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:49.140+0000 7f9ff5ffb700 1 -- 172.21.15.32:0/3109296763 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9fe0011af0 con 0x7f9ff8109a50 2024-05-03T18:36:49.828 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:49.141+0000 7f9ffd918700 1 -- 172.21.15.32:0/3109296763 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ff8109a50 msgr2=0x7f9ff810be40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:36:49.828 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:49.141+0000 7f9ffd918700 1 --2- 172.21.15.32:0/3109296763 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ff8109a50 0x7f9ff810be40 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f9fe00098a0 tx=0x7f9fe0009bb0 comp rx=0 tx=0).stop 2024-05-03T18:36:49.828 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:49.141+0000 7f9ffd918700 1 -- 172.21.15.32:0/3109296763 shutdown_connections 2024-05-03T18:36:49.828 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:49.141+0000 7f9ffd918700 1 --2- 172.21.15.32:0/3109296763 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ff8109a50 0x7f9ff810be40 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:49.828 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:49.141+0000 7f9ffd918700 1 -- 172.21.15.32:0/3109296763 >> 172.21.15.32:0/3109296763 conn(0x7f9ff81034e0 msgr2=0x7f9ff8105900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:36:49.829 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:49.141+0000 7f9ffd918700 1 -- 172.21.15.32:0/3109296763 shutdown_connections 2024-05-03T18:36:49.829 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:49.141+0000 7f9ffd918700 1 -- 172.21.15.32:0/3109296763 wait complete. 2024-05-03T18:36:49.829 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:49.141+0000 7f9ffd918700 1 Processor -- start 2024-05-03T18:36:49.829 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:49.142+0000 7f9ffd918700 1 -- start start 2024-05-03T18:36:49.829 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:49.142+0000 7f9ffd918700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ff8109a50 0x7f9ff81092d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:36:49.829 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:49.142+0000 7f9ffd918700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f9ff810d350 con 0x7f9ff8109a50 2024-05-03T18:36:49.829 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:49.142+0000 7f9ff6ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ff8109a50 0x7f9ff81092d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:36:49.829 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:49.142+0000 7f9ff6ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ff8109a50 0x7f9ff81092d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60846/0 (socket says 172.21.15.32:60846) 2024-05-03T18:36:49.829 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:49.142+0000 7f9ff6ffd700 1 -- 172.21.15.32:0/728694894 learned_addr learned my addr 172.21.15.32:0/728694894 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:36:49.829 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:49.142+0000 7f9ff6ffd700 1 -- 172.21.15.32:0/728694894 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ff8109810 con 0x7f9ff8109a50 2024-05-03T18:36:49.829 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:49.142+0000 7f9ff6ffd700 1 --2- 172.21.15.32:0/728694894 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ff8109a50 0x7f9ff81092d0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f9fe0003f40 tx=0x7f9fe0011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:36:49.829 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:49.142+0000 7f9ffc916700 1 -- 172.21.15.32:0/728694894 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9fe00038b0 con 0x7f9ff8109a50 2024-05-03T18:36:49.829 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:49.142+0000 7f9ffc916700 1 -- 172.21.15.32:0/728694894 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f9fe0003a50 con 0x7f9ff8109a50 2024-05-03T18:36:49.830 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:49.143+0000 7f9ffd918700 1 -- 172.21.15.32:0/728694894 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ff8107a00 con 0x7f9ff8109a50 2024-05-03T18:36:49.830 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:49.143+0000 7f9ffc916700 1 -- 172.21.15.32:0/728694894 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9fe001a5c0 con 0x7f9ff8109a50 2024-05-03T18:36:49.830 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:49.143+0000 7f9ffd918700 1 -- 172.21.15.32:0/728694894 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ff8107f40 con 0x7f9ff8109a50 2024-05-03T18:36:49.830 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:49.143+0000 7f9ffc916700 1 -- 172.21.15.32:0/728694894 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f9fe001ac50 con 0x7f9ff8109a50 2024-05-03T18:36:49.830 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:49.143+0000 7f9ffc916700 1 -- 172.21.15.32:0/728694894 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f9fe000bb10 con 0x7f9ff8109a50 2024-05-03T18:36:49.830 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:49.144+0000 7f9ffd918700 1 -- 172.21.15.32:0/728694894 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ff8064b00 con 0x7f9ff8109a50 2024-05-03T18:36:49.830 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:49.146+0000 7f9ffc916700 1 -- 172.21.15.32:0/728694894 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f9fe000bd10 con 0x7f9ff8109a50 2024-05-03T18:36:49.830 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:49.216+0000 7f9ffd918700 1 -- 172.21.15.32:0/728694894 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f9ff8108240 con 0x7f9ff8109a50 2024-05-03T18:36:49.830 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:49.218+0000 7f9ffc916700 1 -- 172.21.15.32:0/728694894 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f9fe0024430 con 0x7f9ff8109a50 2024-05-03T18:36:49.830 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:49.219+0000 7f9ffd918700 1 -- 172.21.15.32:0/728694894 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ff8109a50 msgr2=0x7f9ff81092d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:36:49.830 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:49.219+0000 7f9ffd918700 1 --2- 172.21.15.32:0/728694894 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ff8109a50 0x7f9ff81092d0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f9fe0003f40 tx=0x7f9fe0011890 comp rx=0 tx=0).stop 2024-05-03T18:36:49.830 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:49.219+0000 7f9ffd918700 1 -- 172.21.15.32:0/728694894 shutdown_connections 2024-05-03T18:36:49.830 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:49.219+0000 7f9ffd918700 1 --2- 172.21.15.32:0/728694894 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ff8109a50 0x7f9ff81092d0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:49.831 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:49.219+0000 7f9ffd918700 1 -- 172.21.15.32:0/728694894 >> 172.21.15.32:0/728694894 conn(0x7f9ff81034e0 msgr2=0x7f9ff8103fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:36:49.831 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:49.219+0000 7f9ffd918700 1 -- 172.21.15.32:0/728694894 shutdown_connections 2024-05-03T18:36:49.831 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:49.219+0000 7f9ffd918700 1 -- 172.21.15.32:0/728694894 wait complete. 2024-05-03T18:36:49.831 INFO:teuthology.orchestra.run.smithi032.stdout:Wrote config to /etc/ceph/ceph.conf 2024-05-03T18:36:49.853 INFO:teuthology.orchestra.run.smithi032.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-05-03T18:36:49.853 INFO:teuthology.orchestra.run.smithi032.stdout:Creating mgr... 2024-05-03T18:36:49.853 INFO:teuthology.orchestra.run.smithi032.stdout:Verifying port 0.0.0.0:9283 ... 2024-05-03T18:36:49.854 INFO:teuthology.orchestra.run.smithi032.stdout:Verifying port 0.0.0.0:8765 ... 2024-05-03T18:36:50.103 INFO:teuthology.orchestra.run.smithi032.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-f500455e-097b-11ef-bc97-c7b262605968@mgr.x 2024-05-03T18:36:50.103 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stderr Failed to reset failed state of unit ceph-f500455e-097b-11ef-bc97-c7b262605968@mgr.x.service: Unit ceph-f500455e-097b-11ef-bc97-c7b262605968@mgr.x.service not loaded. 2024-05-03T18:36:50.255 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-f500455e-097b-11ef-bc97-c7b262605968.target.wants/ceph-f500455e-097b-11ef-bc97-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-f500455e-097b-11ef-bc97-c7b262605968@.service. 2024-05-03T18:36:50.490 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:50 smithi032 ceph-mon[29763]: from='client.? 172.21.15.32:0/728694894' entity='client.admin' 2024-05-03T18:36:50.490 INFO:journalctl@ceph.mgr.x.smithi032.stdout:May 03 18:36:50 smithi032 systemd[1]: Starting Ceph mgr.x for f500455e-097b-11ef-bc97-c7b262605968... 2024-05-03T18:36:50.808 INFO:journalctl@ceph.mgr.x.smithi032.stdout:May 03 18:36:50 smithi032 podman[29980]: 2024-05-03 18:36:50.392747623 +0000 UTC m=+0.018572589 image pull ee4d846202496ba6096593e8e8becc0700a5eb7ebd95536031187f5f337d4012 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea 2024-05-03T18:36:50.808 INFO:journalctl@ceph.mgr.x.smithi032.stdout:May 03 18:36:50 smithi032 podman[29980]: 2024-05-03 18:36:50.480880512 +0000 UTC m=+0.106705505 container create bc042bf2e9b3d95b425de1863e57f1ae61ece100852b9f123efdb71ecd143517 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-x, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240430, GIT_CLEAN=True) 2024-05-03T18:36:50.808 INFO:journalctl@ceph.mgr.x.smithi032.stdout:May 03 18:36:50 smithi032 podman[29980]: 2024-05-03 18:36:50.724662605 +0000 UTC m=+0.350487562 container init bc042bf2e9b3d95b425de1863e57f1ae61ece100852b9f123efdb71ecd143517 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-x, org.label-schema.build-date=20240430, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-05-03T18:36:50.808 INFO:journalctl@ceph.mgr.x.smithi032.stdout:May 03 18:36:50 smithi032 podman[29980]: 2024-05-03 18:36:50.727271616 +0000 UTC m=+0.353096570 container start bc042bf2e9b3d95b425de1863e57f1ae61ece100852b9f123efdb71ecd143517 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-x, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240430, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-03T18:36:50.821 INFO:teuthology.orchestra.run.smithi032.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-03T18:36:50.821 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stdout disabled 2024-05-03T18:36:50.828 INFO:teuthology.orchestra.run.smithi032.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-03T18:36:50.829 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stdout inactive 2024-05-03T18:36:50.829 INFO:teuthology.orchestra.run.smithi032.stdout:firewalld.service is not enabled 2024-05-03T18:36:50.829 INFO:teuthology.orchestra.run.smithi032.stdout:Not possible to enable service . firewalld.service is not available 2024-05-03T18:36:50.837 INFO:teuthology.orchestra.run.smithi032.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-03T18:36:50.837 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stdout disabled 2024-05-03T18:36:50.844 INFO:teuthology.orchestra.run.smithi032.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-03T18:36:50.845 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stdout inactive 2024-05-03T18:36:50.845 INFO:teuthology.orchestra.run.smithi032.stdout:firewalld.service is not enabled 2024-05-03T18:36:50.845 INFO:teuthology.orchestra.run.smithi032.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-05-03T18:36:50.845 INFO:teuthology.orchestra.run.smithi032.stdout:Waiting for mgr to start... 2024-05-03T18:36:50.845 INFO:teuthology.orchestra.run.smithi032.stdout:Waiting for mgr... 2024-05-03T18:36:51.068 INFO:journalctl@ceph.mgr.x.smithi032.stdout:May 03 18:36:50 smithi032 bash[29980]: bc042bf2e9b3d95b425de1863e57f1ae61ece100852b9f123efdb71ecd143517 2024-05-03T18:36:51.068 INFO:journalctl@ceph.mgr.x.smithi032.stdout:May 03 18:36:50 smithi032 systemd[1]: Started Ceph mgr.x for f500455e-097b-11ef-bc97-c7b262605968. 2024-05-03T18:36:51.068 INFO:journalctl@ceph.mgr.x.smithi032.stdout:May 03 18:36:50 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-x[29995]: 2024-05-03T18:36:50.885+0000 7fc81ed94200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-03T18:36:51.398 INFO:journalctl@ceph.mgr.x.smithi032.stdout:May 03 18:36:51 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-x[29995]: 2024-05-03T18:36:51.079+0000 7fc81ed94200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-03T18:36:51.398 INFO:journalctl@ceph.mgr.x.smithi032.stdout:May 03 18:36:51 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-x[29995]: 2024-05-03T18:36:51.189+0000 7fc81ed94200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-03T18:36:51.726 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:51 smithi032 ceph-mon[29763]: from='client.? 172.21.15.32:0/2184688908' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-03T18:36:51.727 INFO:journalctl@ceph.mgr.x.smithi032.stdout:May 03 18:36:51 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-x[29995]: 2024-05-03T18:36:51.540+0000 7fc81ed94200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-03T18:36:51.727 INFO:journalctl@ceph.mgr.x.smithi032.stdout:May 03 18:36:51 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-x[29995]: 2024-05-03T18:36:51.629+0000 7fc81ed94200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-03T18:36:51.985 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout 2024-05-03T18:36:51.985 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout { 2024-05-03T18:36:51.986 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "fsid": "f500455e-097b-11ef-bc97-c7b262605968", 2024-05-03T18:36:51.986 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "health": { 2024-05-03T18:36:51.986 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-03T18:36:51.986 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-03T18:36:51.986 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-03T18:36:51.986 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2024-05-03T18:36:51.987 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-03T18:36:51.987 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-03T18:36:51.987 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout 0 2024-05-03T18:36:51.987 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout ], 2024-05-03T18:36:51.987 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-03T18:36:51.987 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "a" 2024-05-03T18:36:51.987 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout ], 2024-05-03T18:36:51.988 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-05-03T18:36:51.988 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-03T18:36:51.988 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-03T18:36:51.988 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-03T18:36:51.988 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-03T18:36:51.988 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2024-05-03T18:36:51.989 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-03T18:36:51.989 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-03T18:36:51.989 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-03T18:36:51.989 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-03T18:36:51.989 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-03T18:36:51.989 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-03T18:36:51.990 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-03T18:36:51.990 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-03T18:36:51.990 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2024-05-03T18:36:51.990 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-03T18:36:51.990 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-03T18:36:51.990 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-03T18:36:51.990 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-03T18:36:51.991 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-03T18:36:51.991 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-03T18:36:51.991 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-03T18:36:51.991 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-03T18:36:51.991 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-03T18:36:51.991 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2024-05-03T18:36:51.991 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-03T18:36:51.992 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-03T18:36:51.992 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-03T18:36:51.992 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-03T18:36:51.992 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2024-05-03T18:36:51.992 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-03T18:36:51.992 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-03T18:36:51.993 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-03T18:36:51.993 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-03T18:36:51.993 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-03T18:36:51.993 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-03T18:36:51.993 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "restful" 2024-05-03T18:36:51.993 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout ], 2024-05-03T18:36:51.993 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-03T18:36:51.993 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2024-05-03T18:36:51.993 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-03T18:36:51.993 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-03T18:36:51.993 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "modified": "2024-05-03T18:36:43.751410+0000", 2024-05-03T18:36:51.993 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-03T18:36:51.993 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2024-05-03T18:36:51.993 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-03T18:36:51.993 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout } 2024-05-03T18:36:51.994 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:51.296+0000 7fb7d3199700 1 Processor -- start 2024-05-03T18:36:51.994 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:51.296+0000 7fb7d3199700 1 -- start start 2024-05-03T18:36:51.994 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:51.296+0000 7fb7d3199700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7cc07ee90 0x7fb7cc07d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:36:51.994 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:51.296+0000 7fb7d3199700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fb7cc07d8c0 con 0x7fb7cc07ee90 2024-05-03T18:36:51.994 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:51.297+0000 7fb7d0f35700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7cc07ee90 0x7fb7cc07d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:36:51.994 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:51.297+0000 7fb7d0f35700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7cc07ee90 0x7fb7cc07d2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60886/0 (socket says 172.21.15.32:60886) 2024-05-03T18:36:51.994 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:51.297+0000 7fb7d0f35700 1 -- 172.21.15.32:0/1838874773 learned_addr learned my addr 172.21.15.32:0/1838874773 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:36:51.994 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:51.297+0000 7fb7d0f35700 1 -- 172.21.15.32:0/1838874773 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7cc07da40 con 0x7fb7cc07ee90 2024-05-03T18:36:51.994 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:51.297+0000 7fb7d0f35700 1 --2- 172.21.15.32:0/1838874773 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7cc07ee90 0x7fb7cc07d2f0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fb7bc009e80 tx=0x7fb7bc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=428e479d8bc796e5 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:36:51.994 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:51.297+0000 7fb7cb7fe700 1 -- 172.21.15.32:0/1838874773 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb7bc017070 con 0x7fb7cc07ee90 2024-05-03T18:36:51.994 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:51.298+0000 7fb7cb7fe700 1 -- 172.21.15.32:0/1838874773 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fb7bc00b8c0 con 0x7fb7cc07ee90 2024-05-03T18:36:51.994 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:51.298+0000 7fb7cb7fe700 1 -- 172.21.15.32:0/1838874773 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb7bc004490 con 0x7fb7cc07ee90 2024-05-03T18:36:51.994 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:51.298+0000 7fb7d3199700 1 -- 172.21.15.32:0/1838874773 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7cc07ee90 msgr2=0x7fb7cc07d2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:36:51.995 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:51.298+0000 7fb7d3199700 1 --2- 172.21.15.32:0/1838874773 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7cc07ee90 0x7fb7cc07d2f0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fb7bc009e80 tx=0x7fb7bc00b330 comp rx=0 tx=0).stop 2024-05-03T18:36:51.995 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:51.298+0000 7fb7d3199700 1 -- 172.21.15.32:0/1838874773 shutdown_connections 2024-05-03T18:36:51.995 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:51.298+0000 7fb7d3199700 1 --2- 172.21.15.32:0/1838874773 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7cc07ee90 0x7fb7cc07d2f0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:51.995 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:51.298+0000 7fb7d3199700 1 -- 172.21.15.32:0/1838874773 >> 172.21.15.32:0/1838874773 conn(0x7fb7cc104580 msgr2=0x7fb7cc1069a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:36:51.995 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:51.298+0000 7fb7d3199700 1 -- 172.21.15.32:0/1838874773 shutdown_connections 2024-05-03T18:36:51.995 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:51.298+0000 7fb7d3199700 1 -- 172.21.15.32:0/1838874773 wait complete. 2024-05-03T18:36:51.995 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:51.299+0000 7fb7d3199700 1 Processor -- start 2024-05-03T18:36:51.995 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:51.299+0000 7fb7d3199700 1 -- start start 2024-05-03T18:36:51.995 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:51.299+0000 7fb7d3199700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7cc07ee90 0x7fb7cc19f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:36:51.995 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:51.299+0000 7fb7d3199700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fb7cc10f570 con 0x7fb7cc07ee90 2024-05-03T18:36:51.995 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:51.299+0000 7fb7d0f35700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7cc07ee90 0x7fb7cc19f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:36:51.995 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:51.299+0000 7fb7d0f35700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7cc07ee90 0x7fb7cc19f100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60900/0 (socket says 172.21.15.32:60900) 2024-05-03T18:36:51.995 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:51.299+0000 7fb7d0f35700 1 -- 172.21.15.32:0/2184688908 learned_addr learned my addr 172.21.15.32:0/2184688908 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:36:51.995 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:51.299+0000 7fb7d0f35700 1 -- 172.21.15.32:0/2184688908 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7cc19f640 con 0x7fb7cc07ee90 2024-05-03T18:36:51.996 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:51.300+0000 7fb7d0f35700 1 --2- 172.21.15.32:0/2184688908 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7cc07ee90 0x7fb7cc19f100 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb7bc00b7b0 tx=0x7fb7bc00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:36:51.996 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:51.300+0000 7fb7c9ffb700 1 -- 172.21.15.32:0/2184688908 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb7bc020020 con 0x7fb7cc07ee90 2024-05-03T18:36:51.996 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:51.300+0000 7fb7d3199700 1 -- 172.21.15.32:0/2184688908 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7cc19f8d0 con 0x7fb7cc07ee90 2024-05-03T18:36:51.996 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:51.300+0000 7fb7c9ffb700 1 -- 172.21.15.32:0/2184688908 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fb7bc012430 con 0x7fb7cc07ee90 2024-05-03T18:36:51.996 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:51.300+0000 7fb7c9ffb700 1 -- 172.21.15.32:0/2184688908 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb7bc017040 con 0x7fb7cc07ee90 2024-05-03T18:36:51.996 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:51.300+0000 7fb7d3199700 1 -- 172.21.15.32:0/2184688908 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7cc1a25c0 con 0x7fb7cc07ee90 2024-05-03T18:36:51.996 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:51.300+0000 7fb7c9ffb700 1 -- 172.21.15.32:0/2184688908 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fb7bc023020 con 0x7fb7cc07ee90 2024-05-03T18:36:51.996 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:51.300+0000 7fb7c9ffb700 1 -- 172.21.15.32:0/2184688908 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fb7bc00f050 con 0x7fb7cc07ee90 2024-05-03T18:36:51.996 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:51.301+0000 7fb7d3199700 1 -- 172.21.15.32:0/2184688908 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7cc064b00 con 0x7fb7cc07ee90 2024-05-03T18:36:51.996 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:51.304+0000 7fb7c9ffb700 1 -- 172.21.15.32:0/2184688908 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fb7bc02a020 con 0x7fb7cc07ee90 2024-05-03T18:36:51.996 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:51.397+0000 7fb7d3199700 1 -- 172.21.15.32:0/2184688908 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fb7cc1a2ab0 con 0x7fb7cc07ee90 2024-05-03T18:36:51.996 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:51.397+0000 7fb7c9ffb700 1 -- 172.21.15.32:0/2184688908 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fb7bc00b860 con 0x7fb7cc07ee90 2024-05-03T18:36:51.996 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:51.398+0000 7fb7d3199700 1 -- 172.21.15.32:0/2184688908 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7cc07ee90 msgr2=0x7fb7cc19f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:36:51.996 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:51.398+0000 7fb7d3199700 1 --2- 172.21.15.32:0/2184688908 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7cc07ee90 0x7fb7cc19f100 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb7bc00b7b0 tx=0x7fb7bc00be20 comp rx=0 tx=0).stop 2024-05-03T18:36:51.996 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:51.398+0000 7fb7d3199700 1 -- 172.21.15.32:0/2184688908 shutdown_connections 2024-05-03T18:36:51.996 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:51.398+0000 7fb7d3199700 1 --2- 172.21.15.32:0/2184688908 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7cc07ee90 0x7fb7cc19f100 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:51.997 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:51.398+0000 7fb7d3199700 1 -- 172.21.15.32:0/2184688908 >> 172.21.15.32:0/2184688908 conn(0x7fb7cc104580 msgr2=0x7fb7cc105100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:36:51.997 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:51.398+0000 7fb7d3199700 1 -- 172.21.15.32:0/2184688908 shutdown_connections 2024-05-03T18:36:51.997 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:51.398+0000 7fb7d3199700 1 -- 172.21.15.32:0/2184688908 wait complete. 2024-05-03T18:36:51.997 INFO:teuthology.orchestra.run.smithi032.stdout:mgr not available, waiting (1/15)... 2024-05-03T18:36:53.186 INFO:journalctl@ceph.mgr.x.smithi032.stdout:May 03 18:36:52 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-x[29995]: 2024-05-03T18:36:52.899+0000 7fc81ed94200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-03T18:36:53.186 INFO:journalctl@ceph.mgr.x.smithi032.stdout:May 03 18:36:53 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-x[29995]: 2024-05-03T18:36:53.077+0000 7fc81ed94200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-03T18:36:53.557 INFO:journalctl@ceph.mgr.x.smithi032.stdout:May 03 18:36:53 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-x[29995]: 2024-05-03T18:36:53.256+0000 7fc81ed94200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-03T18:36:53.643 INFO:journalctl@ceph.mgr.x.smithi032.stdout:May 03 18:36:53 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-x[29995]: 2024-05-03T18:36:53.556+0000 7fc81ed94200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-03T18:36:53.818 INFO:journalctl@ceph.mgr.x.smithi032.stdout:May 03 18:36:53 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-x[29995]: 2024-05-03T18:36:53.652+0000 7fc81ed94200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-03T18:36:54.215 INFO:journalctl@ceph.mgr.x.smithi032.stdout:May 03 18:36:53 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-x[29995]: 2024-05-03T18:36:53.866+0000 7fc81ed94200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-03T18:36:54.216 INFO:journalctl@ceph.mgr.x.smithi032.stdout:May 03 18:36:54 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-x[29995]: 2024-05-03T18:36:54.073+0000 7fc81ed94200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-03T18:36:54.745 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:54 smithi032 ceph-mon[29763]: from='client.? 172.21.15.32:0/2817542404' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-03T18:36:54.745 INFO:journalctl@ceph.mgr.x.smithi032.stdout:May 03 18:36:54 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-x[29995]: 2024-05-03T18:36:54.722+0000 7fc81ed94200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-03T18:36:54.997 INFO:journalctl@ceph.mgr.x.smithi032.stdout:May 03 18:36:54 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-x[29995]: 2024-05-03T18:36:54.954+0000 7fc81ed94200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-03T18:36:55.000 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout 2024-05-03T18:36:55.000 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout { 2024-05-03T18:36:55.000 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "fsid": "f500455e-097b-11ef-bc97-c7b262605968", 2024-05-03T18:36:55.000 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "health": { 2024-05-03T18:36:55.000 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-03T18:36:55.000 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-03T18:36:55.000 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-03T18:36:55.000 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2024-05-03T18:36:55.000 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-03T18:36:55.000 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-03T18:36:55.000 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout 0 2024-05-03T18:36:55.001 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout ], 2024-05-03T18:36:55.001 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-03T18:36:55.001 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "a" 2024-05-03T18:36:55.001 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout ], 2024-05-03T18:36:55.001 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-05-03T18:36:55.001 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-03T18:36:55.001 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-03T18:36:55.001 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-03T18:36:55.001 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-03T18:36:55.001 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2024-05-03T18:36:55.001 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-03T18:36:55.001 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-03T18:36:55.001 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-03T18:36:55.001 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-03T18:36:55.009 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-03T18:36:55.009 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-03T18:36:55.009 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-03T18:36:55.010 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-03T18:36:55.010 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2024-05-03T18:36:55.010 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-03T18:36:55.010 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-03T18:36:55.010 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-03T18:36:55.010 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-03T18:36:55.010 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-03T18:36:55.010 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-03T18:36:55.010 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-03T18:36:55.010 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-03T18:36:55.010 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-03T18:36:55.010 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2024-05-03T18:36:55.010 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-03T18:36:55.010 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-03T18:36:55.011 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-03T18:36:55.011 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-03T18:36:55.011 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2024-05-03T18:36:55.011 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-03T18:36:55.011 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-03T18:36:55.011 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-03T18:36:55.011 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-03T18:36:55.011 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-03T18:36:55.011 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-03T18:36:55.011 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "restful" 2024-05-03T18:36:55.011 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout ], 2024-05-03T18:36:55.011 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-03T18:36:55.011 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2024-05-03T18:36:55.011 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-03T18:36:55.011 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-03T18:36:55.012 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "modified": "2024-05-03T18:36:43.751410+0000", 2024-05-03T18:36:55.012 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-03T18:36:55.012 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2024-05-03T18:36:55.012 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-03T18:36:55.012 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout } 2024-05-03T18:36:55.012 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:54.409+0000 7fbf73fbb700 1 Processor -- start 2024-05-03T18:36:55.012 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:54.409+0000 7fbf73fbb700 1 -- start start 2024-05-03T18:36:55.012 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:54.410+0000 7fbf73fbb700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf6c109010 0x7fbf6c109430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:36:55.012 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:54.410+0000 7fbf73fbb700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fbf6c109a00 con 0x7fbf6c109010 2024-05-03T18:36:55.013 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:54.410+0000 7fbf71d57700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf6c109010 0x7fbf6c109430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:36:55.013 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:54.410+0000 7fbf71d57700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf6c109010 0x7fbf6c109430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43260/0 (socket says 172.21.15.32:43260) 2024-05-03T18:36:55.013 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:54.410+0000 7fbf71d57700 1 -- 172.21.15.32:0/1166295216 learned_addr learned my addr 172.21.15.32:0/1166295216 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:36:55.013 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:54.410+0000 7fbf71d57700 1 -- 172.21.15.32:0/1166295216 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf6c10a170 con 0x7fbf6c109010 2024-05-03T18:36:55.013 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:54.410+0000 7fbf71d57700 1 --2- 172.21.15.32:0/1166295216 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf6c109010 0x7fbf6c109430 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c009e80 tx=0x7fbf5c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bae1764be4b7e2da server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:36:55.013 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:54.410+0000 7fbf70d55700 1 -- 172.21.15.32:0/1166295216 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbf5c017070 con 0x7fbf6c109010 2024-05-03T18:36:55.013 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:54.410+0000 7fbf70d55700 1 -- 172.21.15.32:0/1166295216 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fbf5c00b8c0 con 0x7fbf6c109010 2024-05-03T18:36:55.013 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:54.411+0000 7fbf70d55700 1 -- 172.21.15.32:0/1166295216 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbf5c004490 con 0x7fbf6c109010 2024-05-03T18:36:55.013 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:54.411+0000 7fbf73fbb700 1 -- 172.21.15.32:0/1166295216 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf6c109010 msgr2=0x7fbf6c109430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:36:55.013 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:54.411+0000 7fbf73fbb700 1 --2- 172.21.15.32:0/1166295216 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf6c109010 0x7fbf6c109430 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c009e80 tx=0x7fbf5c00b330 comp rx=0 tx=0).stop 2024-05-03T18:36:55.013 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:54.411+0000 7fbf73fbb700 1 -- 172.21.15.32:0/1166295216 shutdown_connections 2024-05-03T18:36:55.013 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:54.411+0000 7fbf73fbb700 1 --2- 172.21.15.32:0/1166295216 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf6c109010 0x7fbf6c109430 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:55.014 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:54.411+0000 7fbf73fbb700 1 -- 172.21.15.32:0/1166295216 >> 172.21.15.32:0/1166295216 conn(0x7fbf6c104650 msgr2=0x7fbf6c106a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:36:55.014 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:54.411+0000 7fbf73fbb700 1 -- 172.21.15.32:0/1166295216 shutdown_connections 2024-05-03T18:36:55.014 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:54.411+0000 7fbf73fbb700 1 -- 172.21.15.32:0/1166295216 wait complete. 2024-05-03T18:36:55.014 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:54.412+0000 7fbf73fbb700 1 Processor -- start 2024-05-03T18:36:55.014 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:54.412+0000 7fbf73fbb700 1 -- start start 2024-05-03T18:36:55.014 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:54.412+0000 7fbf73fbb700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf6c109010 0x7fbf6c07eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:36:55.014 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:54.412+0000 7fbf73fbb700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fbf6c10d520 con 0x7fbf6c109010 2024-05-03T18:36:55.014 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:54.412+0000 7fbf71d57700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf6c109010 0x7fbf6c07eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:36:55.014 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:54.412+0000 7fbf71d57700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf6c109010 0x7fbf6c07eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43272/0 (socket says 172.21.15.32:43272) 2024-05-03T18:36:55.014 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:54.412+0000 7fbf71d57700 1 -- 172.21.15.32:0/2817542404 learned_addr learned my addr 172.21.15.32:0/2817542404 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:36:55.014 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:54.412+0000 7fbf71d57700 1 -- 172.21.15.32:0/2817542404 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf6c07f0c0 con 0x7fbf6c109010 2024-05-03T18:36:55.014 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:54.413+0000 7fbf71d57700 1 --2- 172.21.15.32:0/2817542404 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf6c109010 0x7fbf6c07eb80 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c004a60 tx=0x7fbf5c00bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:36:55.014 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:54.413+0000 7fbf6affd700 1 -- 172.21.15.32:0/2817542404 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbf5c020020 con 0x7fbf6c109010 2024-05-03T18:36:55.014 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:54.413+0000 7fbf6affd700 1 -- 172.21.15.32:0/2817542404 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fbf5c012430 con 0x7fbf6c109010 2024-05-03T18:36:55.015 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:54.413+0000 7fbf6affd700 1 -- 172.21.15.32:0/2817542404 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbf5c017040 con 0x7fbf6c109010 2024-05-03T18:36:55.015 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:54.413+0000 7fbf73fbb700 1 -- 172.21.15.32:0/2817542404 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf6c07d2b0 con 0x7fbf6c109010 2024-05-03T18:36:55.015 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:54.413+0000 7fbf73fbb700 1 -- 172.21.15.32:0/2817542404 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf6c07d7f0 con 0x7fbf6c109010 2024-05-03T18:36:55.015 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:54.413+0000 7fbf6affd700 1 -- 172.21.15.32:0/2817542404 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fbf5c023020 con 0x7fbf6c109010 2024-05-03T18:36:55.015 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:54.413+0000 7fbf6affd700 1 -- 172.21.15.32:0/2817542404 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fbf5c00f050 con 0x7fbf6c109010 2024-05-03T18:36:55.015 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:54.414+0000 7fbf73fbb700 1 -- 172.21.15.32:0/2817542404 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf6c064b00 con 0x7fbf6c109010 2024-05-03T18:36:55.015 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:54.416+0000 7fbf6affd700 1 -- 172.21.15.32:0/2817542404 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fbf5c02a020 con 0x7fbf6c109010 2024-05-03T18:36:55.015 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:54.490+0000 7fbf73fbb700 1 -- 172.21.15.32:0/2817542404 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fbf6c07da40 con 0x7fbf6c109010 2024-05-03T18:36:55.015 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:54.491+0000 7fbf6affd700 1 -- 172.21.15.32:0/2817542404 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fbf5c025d50 con 0x7fbf6c109010 2024-05-03T18:36:55.015 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:54.492+0000 7fbf73fbb700 1 -- 172.21.15.32:0/2817542404 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf6c109010 msgr2=0x7fbf6c07eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:36:55.015 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:54.492+0000 7fbf73fbb700 1 --2- 172.21.15.32:0/2817542404 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf6c109010 0x7fbf6c07eb80 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c004a60 tx=0x7fbf5c00bd50 comp rx=0 tx=0).stop 2024-05-03T18:36:55.015 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:54.492+0000 7fbf73fbb700 1 -- 172.21.15.32:0/2817542404 shutdown_connections 2024-05-03T18:36:55.015 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:54.492+0000 7fbf73fbb700 1 --2- 172.21.15.32:0/2817542404 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf6c109010 0x7fbf6c07eb80 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:55.016 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:54.492+0000 7fbf73fbb700 1 -- 172.21.15.32:0/2817542404 >> 172.21.15.32:0/2817542404 conn(0x7fbf6c104650 msgr2=0x7fbf6c105140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:36:55.016 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:54.493+0000 7fbf73fbb700 1 -- 172.21.15.32:0/2817542404 shutdown_connections 2024-05-03T18:36:55.016 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:54.493+0000 7fbf73fbb700 1 -- 172.21.15.32:0/2817542404 wait complete. 2024-05-03T18:36:55.016 INFO:teuthology.orchestra.run.smithi032.stdout:mgr not available, waiting (2/15)... 2024-05-03T18:36:55.817 INFO:journalctl@ceph.mgr.x.smithi032.stdout:May 03 18:36:55 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-x[29995]: 2024-05-03T18:36:55.397+0000 7fc81ed94200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-03T18:36:55.818 INFO:journalctl@ceph.mgr.x.smithi032.stdout:May 03 18:36:55 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-x[29995]: 2024-05-03T18:36:55.496+0000 7fc81ed94200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-03T18:36:56.254 INFO:journalctl@ceph.mgr.x.smithi032.stdout:May 03 18:36:55 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-x[29995]: 2024-05-03T18:36:55.880+0000 7fc81ed94200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-03T18:36:56.254 INFO:journalctl@ceph.mgr.x.smithi032.stdout:May 03 18:36:56 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-x[29995]: 2024-05-03T18:36:56.253+0000 7fc81ed94200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-03T18:36:56.567 INFO:journalctl@ceph.mgr.x.smithi032.stdout:May 03 18:36:56 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-x[29995]: 2024-05-03T18:36:56.348+0000 7fc81ed94200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-03T18:36:57.060 INFO:journalctl@ceph.mgr.x.smithi032.stdout:May 03 18:36:56 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-x[29995]: 2024-05-03T18:36:56.601+0000 7fc81ed94200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-03T18:36:57.060 INFO:journalctl@ceph.mgr.x.smithi032.stdout:May 03 18:36:56 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-x[29995]: 2024-05-03T18:36:56.740+0000 7fc81ed94200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-03T18:36:57.061 INFO:journalctl@ceph.mgr.x.smithi032.stdout:May 03 18:36:57 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-x[29995]: 2024-05-03T18:36:57.060+0000 7fc81ed94200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-03T18:36:57.317 INFO:journalctl@ceph.mgr.x.smithi032.stdout:May 03 18:36:57 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-x[29995]: 2024-05-03T18:36:57.180+0000 7fc81ed94200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-03T18:36:57.318 INFO:journalctl@ceph.mgr.x.smithi032.stdout:May 03 18:36:57 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-x[29995]: 2024-05-03T18:36:57.280+0000 7fc81ed94200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-03T18:36:57.817 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:57 smithi032 ceph-mon[29763]: from='client.? 172.21.15.32:0/4178890853' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-03T18:36:58.126 INFO:journalctl@ceph.mgr.x.smithi032.stdout:May 03 18:36:57 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-x[29995]: 2024-05-03T18:36:57.921+0000 7fc81ed94200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-03T18:36:58.131 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout 2024-05-03T18:36:58.132 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout { 2024-05-03T18:36:58.132 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "fsid": "f500455e-097b-11ef-bc97-c7b262605968", 2024-05-03T18:36:58.132 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "health": { 2024-05-03T18:36:58.132 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-03T18:36:58.132 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-03T18:36:58.132 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-03T18:36:58.132 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2024-05-03T18:36:58.132 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-03T18:36:58.132 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-03T18:36:58.132 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout 0 2024-05-03T18:36:58.132 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout ], 2024-05-03T18:36:58.132 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-03T18:36:58.133 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "a" 2024-05-03T18:36:58.133 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout ], 2024-05-03T18:36:58.133 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2024-05-03T18:36:58.133 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-03T18:36:58.133 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-03T18:36:58.133 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-03T18:36:58.133 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-03T18:36:58.133 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2024-05-03T18:36:58.133 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-03T18:36:58.133 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-03T18:36:58.133 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-03T18:36:58.133 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-03T18:36:58.133 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-03T18:36:58.143 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-03T18:36:58.143 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-03T18:36:58.143 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-03T18:36:58.143 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2024-05-03T18:36:58.143 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-03T18:36:58.143 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-03T18:36:58.143 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-03T18:36:58.143 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-03T18:36:58.143 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-03T18:36:58.143 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-03T18:36:58.144 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-03T18:36:58.144 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-03T18:36:58.144 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-03T18:36:58.144 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2024-05-03T18:36:58.144 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-03T18:36:58.144 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-03T18:36:58.144 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-03T18:36:58.144 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-03T18:36:58.144 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2024-05-03T18:36:58.144 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-03T18:36:58.144 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-03T18:36:58.145 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-03T18:36:58.145 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-03T18:36:58.145 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-03T18:36:58.145 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-03T18:36:58.145 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "restful" 2024-05-03T18:36:58.145 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout ], 2024-05-03T18:36:58.145 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-03T18:36:58.145 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2024-05-03T18:36:58.145 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-03T18:36:58.145 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-03T18:36:58.145 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "modified": "2024-05-03T18:36:43.751410+0000", 2024-05-03T18:36:58.145 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-03T18:36:58.146 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2024-05-03T18:36:58.146 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-03T18:36:58.146 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout } 2024-05-03T18:36:58.146 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:57.431+0000 7f36f67a0700 1 Processor -- start 2024-05-03T18:36:58.146 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:57.431+0000 7f36f67a0700 1 -- start start 2024-05-03T18:36:58.146 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:57.431+0000 7f36f67a0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36f0107e90 0x7f36f01082b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:36:58.146 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:57.431+0000 7f36f67a0700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f36f01087f0 con 0x7f36f0107e90 2024-05-03T18:36:58.146 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:57.431+0000 7f36effff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36f0107e90 0x7f36f01082b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:36:58.146 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:57.431+0000 7f36effff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36f0107e90 0x7f36f01082b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43280/0 (socket says 172.21.15.32:43280) 2024-05-03T18:36:58.146 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:57.431+0000 7f36effff700 1 -- 172.21.15.32:0/2569068427 learned_addr learned my addr 172.21.15.32:0/2569068427 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:36:58.146 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:57.431+0000 7f36effff700 1 -- 172.21.15.32:0/2569068427 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36f0108ff0 con 0x7f36f0107e90 2024-05-03T18:36:58.146 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:57.432+0000 7f36effff700 1 --2- 172.21.15.32:0/2569068427 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36f0107e90 0x7f36f01082b0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f36e00098a0 tx=0x7f36e0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18d962deec6f0c9a server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:36:58.147 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:57.432+0000 7f36eeffd700 1 -- 172.21.15.32:0/2569068427 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f36e0004160 con 0x7f36f0107e90 2024-05-03T18:36:58.147 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:57.432+0000 7f36eeffd700 1 -- 172.21.15.32:0/2569068427 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f36e0004300 con 0x7f36f0107e90 2024-05-03T18:36:58.147 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:57.432+0000 7f36eeffd700 1 -- 172.21.15.32:0/2569068427 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f36e0011af0 con 0x7f36f0107e90 2024-05-03T18:36:58.147 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:57.432+0000 7f36f67a0700 1 -- 172.21.15.32:0/2569068427 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36f0107e90 msgr2=0x7f36f01082b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:36:58.147 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:57.432+0000 7f36f67a0700 1 --2- 172.21.15.32:0/2569068427 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36f0107e90 0x7f36f01082b0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f36e00098a0 tx=0x7f36e0009bb0 comp rx=0 tx=0).stop 2024-05-03T18:36:58.147 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:57.432+0000 7f36f67a0700 1 -- 172.21.15.32:0/2569068427 shutdown_connections 2024-05-03T18:36:58.147 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:57.432+0000 7f36f67a0700 1 --2- 172.21.15.32:0/2569068427 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36f0107e90 0x7f36f01082b0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:58.148 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:57.432+0000 7f36f67a0700 1 -- 172.21.15.32:0/2569068427 >> 172.21.15.32:0/2569068427 conn(0x7f36f01034d0 msgr2=0x7f36f01058f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:36:58.148 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:57.432+0000 7f36f67a0700 1 -- 172.21.15.32:0/2569068427 shutdown_connections 2024-05-03T18:36:58.148 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:57.433+0000 7f36f67a0700 1 -- 172.21.15.32:0/2569068427 wait complete. 2024-05-03T18:36:58.148 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:57.433+0000 7f36f67a0700 1 Processor -- start 2024-05-03T18:36:58.148 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:57.433+0000 7f36f67a0700 1 -- start start 2024-05-03T18:36:58.148 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:57.434+0000 7f36f67a0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36f0107e90 0x7f36f019bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:36:58.148 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:57.434+0000 7f36f67a0700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f36f010c430 con 0x7f36f0107e90 2024-05-03T18:36:58.148 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:57.434+0000 7f36effff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36f0107e90 0x7f36f019bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:36:58.148 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:57.434+0000 7f36effff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36f0107e90 0x7f36f019bde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43292/0 (socket says 172.21.15.32:43292) 2024-05-03T18:36:58.148 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:57.434+0000 7f36effff700 1 -- 172.21.15.32:0/4178890853 learned_addr learned my addr 172.21.15.32:0/4178890853 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:36:58.148 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:57.434+0000 7f36effff700 1 -- 172.21.15.32:0/4178890853 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36f019c320 con 0x7f36f0107e90 2024-05-03T18:36:58.148 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:57.434+0000 7f36effff700 1 --2- 172.21.15.32:0/4178890853 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36f0107e90 0x7f36f019bde0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f36e0011c00 tx=0x7f36e0011ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:36:58.149 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:57.434+0000 7f36ed7fa700 1 -- 172.21.15.32:0/4178890853 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f36e0003930 con 0x7f36f0107e90 2024-05-03T18:36:58.149 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:57.435+0000 7f36ed7fa700 1 -- 172.21.15.32:0/4178890853 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f36e0003ad0 con 0x7f36f0107e90 2024-05-03T18:36:58.149 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:57.435+0000 7f36ed7fa700 1 -- 172.21.15.32:0/4178890853 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f36e001c780 con 0x7f36f0107e90 2024-05-03T18:36:58.149 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:57.435+0000 7f36f67a0700 1 -- 172.21.15.32:0/4178890853 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36f019c5b0 con 0x7f36f0107e90 2024-05-03T18:36:58.149 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:57.435+0000 7f36f67a0700 1 -- 172.21.15.32:0/4178890853 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36f0075ec0 con 0x7f36f0107e90 2024-05-03T18:36:58.149 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:57.435+0000 7f36ed7fa700 1 -- 172.21.15.32:0/4178890853 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f36e001c9f0 con 0x7f36f0107e90 2024-05-03T18:36:58.149 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:57.435+0000 7f36ed7fa700 1 -- 172.21.15.32:0/4178890853 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f36e0025770 con 0x7f36f0107e90 2024-05-03T18:36:58.149 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:57.436+0000 7f36f67a0700 1 -- 172.21.15.32:0/4178890853 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36f0064b00 con 0x7f36f0107e90 2024-05-03T18:36:58.149 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:57.438+0000 7f36ed7fa700 1 -- 172.21.15.32:0/4178890853 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f36e0044e50 con 0x7f36f0107e90 2024-05-03T18:36:58.149 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:57.512+0000 7f36f67a0700 1 -- 172.21.15.32:0/4178890853 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f36f0076460 con 0x7f36f0107e90 2024-05-03T18:36:58.149 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:57.512+0000 7f36ed7fa700 1 -- 172.21.15.32:0/4178890853 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f36e0033030 con 0x7f36f0107e90 2024-05-03T18:36:58.149 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:57.513+0000 7f36f67a0700 1 -- 172.21.15.32:0/4178890853 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36f0107e90 msgr2=0x7f36f019bde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:36:58.150 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:57.513+0000 7f36f67a0700 1 --2- 172.21.15.32:0/4178890853 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36f0107e90 0x7f36f019bde0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f36e0011c00 tx=0x7f36e0011ce0 comp rx=0 tx=0).stop 2024-05-03T18:36:58.150 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:57.513+0000 7f36f67a0700 1 -- 172.21.15.32:0/4178890853 shutdown_connections 2024-05-03T18:36:58.150 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:57.513+0000 7f36f67a0700 1 --2- 172.21.15.32:0/4178890853 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36f0107e90 0x7f36f019bde0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:36:58.150 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:57.513+0000 7f36f67a0700 1 -- 172.21.15.32:0/4178890853 >> 172.21.15.32:0/4178890853 conn(0x7f36f01034d0 msgr2=0x7f36f0103fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:36:58.150 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:57.513+0000 7f36f67a0700 1 -- 172.21.15.32:0/4178890853 shutdown_connections 2024-05-03T18:36:58.150 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:36:57.513+0000 7f36f67a0700 1 -- 172.21.15.32:0/4178890853 wait complete. 2024-05-03T18:36:58.150 INFO:teuthology.orchestra.run.smithi032.stdout:mgr not available, waiting (3/15)... 2024-05-03T18:36:58.564 INFO:journalctl@ceph.mgr.x.smithi032.stdout:May 03 18:36:58 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-x[29995]: 2024-05-03T18:36:58.147+0000 7fc81ed94200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-03T18:36:58.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:58 smithi032 ceph-mon[29763]: Activating manager daemon x 2024-05-03T18:36:58.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:58 smithi032 ceph-mon[29763]: mgrmap e2: x(active, starting, since 0.00310742s) 2024-05-03T18:36:58.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:58 smithi032 ceph-mon[29763]: from='mgr.14100 172.21.15.32:0/1920787905' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-03T18:36:58.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:58 smithi032 ceph-mon[29763]: from='mgr.14100 172.21.15.32:0/1920787905' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-03T18:36:58.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:58 smithi032 ceph-mon[29763]: from='mgr.14100 172.21.15.32:0/1920787905' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-03T18:36:58.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:58 smithi032 ceph-mon[29763]: from='mgr.14100 172.21.15.32:0/1920787905' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-03T18:36:58.819 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:58 smithi032 ceph-mon[29763]: from='mgr.14100 172.21.15.32:0/1920787905' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-03T18:36:58.819 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:58 smithi032 ceph-mon[29763]: Manager daemon x is now available 2024-05-03T18:36:58.819 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:58 smithi032 ceph-mon[29763]: from='mgr.14100 172.21.15.32:0/1920787905' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-03T18:36:58.819 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:58 smithi032 ceph-mon[29763]: from='mgr.14100 172.21.15.32:0/1920787905' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-03T18:36:58.819 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:58 smithi032 ceph-mon[29763]: from='mgr.14100 172.21.15.32:0/1920787905' entity='mgr.x' 2024-05-03T18:36:58.819 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:58 smithi032 ceph-mon[29763]: from='mgr.14100 172.21.15.32:0/1920787905' entity='mgr.x' 2024-05-03T18:36:58.819 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:36:58 smithi032 ceph-mon[29763]: from='mgr.14100 172.21.15.32:0/1920787905' entity='mgr.x' 2024-05-03T18:37:00.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:37:00 smithi032 ceph-mon[29763]: mgrmap e3: x(active, since 1.00793s) 2024-05-03T18:37:01.567 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:37:01 smithi032 ceph-mon[29763]: mgrmap e4: x(active, since 2s) 2024-05-03T18:37:04.495 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:37:04 smithi032 ceph-mon[29763]: from='client.? 172.21.15.32:0/2696088645' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-03T18:37:09.314 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout 2024-05-03T18:37:09.314 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout { 2024-05-03T18:37:09.314 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "fsid": "f500455e-097b-11ef-bc97-c7b262605968", 2024-05-03T18:37:09.314 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "health": { 2024-05-03T18:37:09.314 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-03T18:37:09.314 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-03T18:37:09.314 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-03T18:37:09.314 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2024-05-03T18:37:09.314 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-03T18:37:09.314 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-03T18:37:09.315 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout 0 2024-05-03T18:37:09.315 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout ], 2024-05-03T18:37:09.315 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-03T18:37:09.315 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "a" 2024-05-03T18:37:09.315 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout ], 2024-05-03T18:37:09.315 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "quorum_age": 15, 2024-05-03T18:37:09.315 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-03T18:37:09.315 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-03T18:37:09.315 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-03T18:37:09.315 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-03T18:37:09.315 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2024-05-03T18:37:09.315 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-03T18:37:09.315 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-03T18:37:09.315 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-03T18:37:09.315 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-03T18:37:09.316 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-03T18:37:09.316 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-03T18:37:09.316 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-03T18:37:09.316 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-03T18:37:09.316 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2024-05-03T18:37:09.316 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-03T18:37:09.316 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-03T18:37:09.316 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-03T18:37:09.316 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-03T18:37:09.316 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-03T18:37:09.316 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-03T18:37:09.316 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-03T18:37:09.316 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-03T18:37:09.316 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-03T18:37:09.316 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2024-05-03T18:37:09.316 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-03T18:37:09.317 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-03T18:37:09.317 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-03T18:37:09.317 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-03T18:37:09.317 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2024-05-03T18:37:09.317 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-03T18:37:09.317 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-03T18:37:09.317 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-03T18:37:09.317 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-03T18:37:09.317 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-03T18:37:09.317 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-03T18:37:09.317 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "restful" 2024-05-03T18:37:09.318 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout ], 2024-05-03T18:37:09.318 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-03T18:37:09.318 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2024-05-03T18:37:09.318 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-03T18:37:09.318 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-03T18:37:09.318 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "modified": "2024-05-03T18:36:43.751410+0000", 2024-05-03T18:37:09.318 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-03T18:37:09.318 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout }, 2024-05-03T18:37:09.318 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-03T18:37:09.318 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout } 2024-05-03T18:37:09.318 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:03.888+0000 7f99b2aa6700 1 Processor -- start 2024-05-03T18:37:09.318 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:03.888+0000 7f99b2aa6700 1 -- start start 2024-05-03T18:37:09.318 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:03.888+0000 7f99b2aa6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99ac107850 0x7f99ac107c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:09.318 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:03.888+0000 7f99b2aa6700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f99ac07ce60 con 0x7f99ac107850 2024-05-03T18:37:09.319 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:03.888+0000 7f99b0842700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99ac107850 0x7f99ac107c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:09.319 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:03.888+0000 7f99b0842700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99ac107850 0x7f99ac107c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52510/0 (socket says 172.21.15.32:52510) 2024-05-03T18:37:09.319 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:03.888+0000 7f99b0842700 1 -- 172.21.15.32:0/3109312036 learned_addr learned my addr 172.21.15.32:0/3109312036 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:37:09.319 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:03.889+0000 7f99b0842700 1 -- 172.21.15.32:0/3109312036 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99ac1081b0 con 0x7f99ac107850 2024-05-03T18:37:09.319 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:03.889+0000 7f99b0842700 1 --2- 172.21.15.32:0/3109312036 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99ac107850 0x7f99ac107c70 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f99a00098a0 tx=0x7f99a0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bf179b82516f726 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:37:09.319 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:03.889+0000 7f99ab7fe700 1 -- 172.21.15.32:0/3109312036 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f99a0004160 con 0x7f99ac107850 2024-05-03T18:37:09.319 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:03.889+0000 7f99ab7fe700 1 -- 172.21.15.32:0/3109312036 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f99a0004300 con 0x7f99ac107850 2024-05-03T18:37:09.319 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:03.889+0000 7f99ab7fe700 1 -- 172.21.15.32:0/3109312036 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f99a0011af0 con 0x7f99ac107850 2024-05-03T18:37:09.319 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:03.890+0000 7f99b2aa6700 1 -- 172.21.15.32:0/3109312036 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99ac107850 msgr2=0x7f99ac107c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:37:09.319 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:03.890+0000 7f99b2aa6700 1 --2- 172.21.15.32:0/3109312036 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99ac107850 0x7f99ac107c70 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f99a00098a0 tx=0x7f99a0009bb0 comp rx=0 tx=0).stop 2024-05-03T18:37:09.319 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:03.890+0000 7f99b2aa6700 1 -- 172.21.15.32:0/3109312036 shutdown_connections 2024-05-03T18:37:09.319 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:03.890+0000 7f99b2aa6700 1 --2- 172.21.15.32:0/3109312036 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99ac107850 0x7f99ac107c70 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:09.319 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:03.890+0000 7f99b2aa6700 1 -- 172.21.15.32:0/3109312036 >> 172.21.15.32:0/3109312036 conn(0x7f99ac1034d0 msgr2=0x7f99ac1058f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:37:09.319 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:03.890+0000 7f99b2aa6700 1 -- 172.21.15.32:0/3109312036 shutdown_connections 2024-05-03T18:37:09.319 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:03.890+0000 7f99b2aa6700 1 -- 172.21.15.32:0/3109312036 wait complete. 2024-05-03T18:37:09.320 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:03.890+0000 7f99b2aa6700 1 Processor -- start 2024-05-03T18:37:09.320 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:03.891+0000 7f99b2aa6700 1 -- start start 2024-05-03T18:37:09.320 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:03.891+0000 7f99b2aa6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99ac107850 0x7f99ac19f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:09.320 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:03.891+0000 7f99b2aa6700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f99ac1091e0 con 0x7f99ac107850 2024-05-03T18:37:09.320 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:03.891+0000 7f99b0842700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99ac107850 0x7f99ac19f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:09.320 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:03.891+0000 7f99b0842700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99ac107850 0x7f99ac19f140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52522/0 (socket says 172.21.15.32:52522) 2024-05-03T18:37:09.320 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:03.891+0000 7f99b0842700 1 -- 172.21.15.32:0/2696088645 learned_addr learned my addr 172.21.15.32:0/2696088645 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:37:09.320 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:03.891+0000 7f99b0842700 1 -- 172.21.15.32:0/2696088645 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99ac19f680 con 0x7f99ac107850 2024-05-03T18:37:09.320 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:03.891+0000 7f99b0842700 1 --2- 172.21.15.32:0/2696088645 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99ac107850 0x7f99ac19f140 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f99a0011ac0 tx=0x7f99a0011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:37:09.320 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:03.891+0000 7f99a9ffb700 1 -- 172.21.15.32:0/2696088645 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f99a0003970 con 0x7f99ac107850 2024-05-03T18:37:09.320 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:03.891+0000 7f99a9ffb700 1 -- 172.21.15.32:0/2696088645 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f99a0003b10 con 0x7f99ac107850 2024-05-03T18:37:09.320 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:03.891+0000 7f99b2aa6700 1 -- 172.21.15.32:0/2696088645 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99ac19f910 con 0x7f99ac107850 2024-05-03T18:37:09.320 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:03.892+0000 7f99a9ffb700 1 -- 172.21.15.32:0/2696088645 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f99a0024610 con 0x7f99ac107850 2024-05-03T18:37:09.320 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:03.892+0000 7f99b2aa6700 1 -- 172.21.15.32:0/2696088645 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99ac1a2600 con 0x7f99ac107850 2024-05-03T18:37:09.321 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:03.892+0000 7f99a9ffb700 1 -- 172.21.15.32:0/2696088645 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) ==== 50211+0+0 (secure 0 0 0) 0x7f99a0024890 con 0x7f99ac107850 2024-05-03T18:37:09.321 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:03.893+0000 7f99b2aa6700 1 -- 172.21.15.32:0/2696088645 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99ac064b00 con 0x7f99ac107850 2024-05-03T18:37:09.321 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:03.893+0000 7f99a9ffb700 1 --2- 172.21.15.32:0/2696088645 >> [v2:172.21.15.32:6800/3945405684,v1:172.21.15.32:6801/3945405684] conn(0x7f999c03e420 0x7f999c0408e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:09.321 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:03.893+0000 7f99a9ffb700 1 -- 172.21.15.32:0/2696088645 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f99a00547b0 con 0x7f99ac107850 2024-05-03T18:37:09.321 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:03.893+0000 7f99abfff700 1 --2- 172.21.15.32:0/2696088645 >> [v2:172.21.15.32:6800/3945405684,v1:172.21.15.32:6801/3945405684] conn(0x7f999c03e420 0x7f999c0408e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:09.321 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:03.893+0000 7f99abfff700 1 --2- 172.21.15.32:0/2696088645 >> [v2:172.21.15.32:6800/3945405684,v1:172.21.15.32:6801/3945405684] conn(0x7f999c03e420 0x7f999c0408e0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f9998006fd0 tx=0x7f9998006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:37:09.321 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:03.898+0000 7f99a9ffb700 1 -- 172.21.15.32:0/2696088645 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f99a002c350 con 0x7f99ac107850 2024-05-03T18:37:09.321 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:04.209+0000 7f99b2aa6700 1 -- 172.21.15.32:0/2696088645 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f99ac0008d0 con 0x7f99ac107850 2024-05-03T18:37:09.321 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:04.210+0000 7f99a9ffb700 1 -- 172.21.15.32:0/2696088645 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f99a00172b0 con 0x7f99ac107850 2024-05-03T18:37:09.321 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:04.212+0000 7f99b2aa6700 1 -- 172.21.15.32:0/2696088645 >> [v2:172.21.15.32:6800/3945405684,v1:172.21.15.32:6801/3945405684] conn(0x7f999c03e420 msgr2=0x7f999c0408e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:37:09.321 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:04.212+0000 7f99b2aa6700 1 --2- 172.21.15.32:0/2696088645 >> [v2:172.21.15.32:6800/3945405684,v1:172.21.15.32:6801/3945405684] conn(0x7f999c03e420 0x7f999c0408e0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f9998006fd0 tx=0x7f9998006e40 comp rx=0 tx=0).stop 2024-05-03T18:37:09.321 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:04.212+0000 7f99b2aa6700 1 -- 172.21.15.32:0/2696088645 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99ac107850 msgr2=0x7f99ac19f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:37:09.321 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:04.212+0000 7f99b2aa6700 1 --2- 172.21.15.32:0/2696088645 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99ac107850 0x7f99ac19f140 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f99a0011ac0 tx=0x7f99a0011ba0 comp rx=0 tx=0).stop 2024-05-03T18:37:09.321 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:04.212+0000 7f99b2aa6700 1 -- 172.21.15.32:0/2696088645 shutdown_connections 2024-05-03T18:37:09.321 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:04.212+0000 7f99b2aa6700 1 --2- 172.21.15.32:0/2696088645 >> [v2:172.21.15.32:6800/3945405684,v1:172.21.15.32:6801/3945405684] conn(0x7f999c03e420 0x7f999c0408e0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:09.322 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:04.212+0000 7f99b2aa6700 1 --2- 172.21.15.32:0/2696088645 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99ac107850 0x7f99ac19f140 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:09.322 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:04.212+0000 7f99b2aa6700 1 -- 172.21.15.32:0/2696088645 >> 172.21.15.32:0/2696088645 conn(0x7f99ac1034d0 msgr2=0x7f99ac195e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:37:09.322 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:04.213+0000 7f99b2aa6700 1 -- 172.21.15.32:0/2696088645 shutdown_connections 2024-05-03T18:37:09.322 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:04.213+0000 7f99b2aa6700 1 -- 172.21.15.32:0/2696088645 wait complete. 2024-05-03T18:37:09.322 INFO:teuthology.orchestra.run.smithi032.stdout:mgr is available 2024-05-03T18:37:10.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:37:09 smithi032 ceph-mon[29763]: from='client.? 172.21.15.32:0/4261131945' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-05-03T18:37:10.448 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout 2024-05-03T18:37:10.449 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout [global] 2024-05-03T18:37:10.449 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout fsid = f500455e-097b-11ef-bc97-c7b262605968 2024-05-03T18:37:10.449 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-05-03T18:37:10.449 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-05-03T18:37:10.449 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-05-03T18:37:10.449 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-05-03T18:37:10.449 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-05-03T18:37:10.449 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout 2024-05-03T18:37:10.449 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout [mgr] 2024-05-03T18:37:10.449 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-05-03T18:37:10.449 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout 2024-05-03T18:37:10.449 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout [osd] 2024-05-03T18:37:10.449 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout bdev_async_discard = True 2024-05-03T18:37:10.449 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-05-03T18:37:10.450 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-05-03T18:37:10.450 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-05-03T18:37:10.450 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-05-03T18:37:10.450 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-05-03T18:37:10.450 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:09.720+0000 7fcd541c5700 1 Processor -- start 2024-05-03T18:37:10.450 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:09.720+0000 7fcd541c5700 1 -- start start 2024-05-03T18:37:10.450 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:09.720+0000 7fcd541c5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd4c07ee10 0x7fcd4c07d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:10.450 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:09.720+0000 7fcd541c5700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fcd4c07d850 con 0x7fcd4c07ee10 2024-05-03T18:37:10.450 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:09.721+0000 7fcd51f61700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd4c07ee10 0x7fcd4c07d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:10.450 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:09.721+0000 7fcd51f61700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd4c07ee10 0x7fcd4c07d310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52524/0 (socket says 172.21.15.32:52524) 2024-05-03T18:37:10.450 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:09.721+0000 7fcd51f61700 1 -- 172.21.15.32:0/1737766658 learned_addr learned my addr 172.21.15.32:0/1737766658 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:37:10.450 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:09.721+0000 7fcd51f61700 1 -- 172.21.15.32:0/1737766658 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd4c07d9d0 con 0x7fcd4c07ee10 2024-05-03T18:37:10.450 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:09.721+0000 7fcd51f61700 1 --2- 172.21.15.32:0/1737766658 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd4c07ee10 0x7fcd4c07d310 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fcd44009e80 tx=0x7fcd4400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e6f2edbf11fe987 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:37:10.451 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:09.721+0000 7fcd50f5f700 1 -- 172.21.15.32:0/1737766658 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcd44017070 con 0x7fcd4c07ee10 2024-05-03T18:37:10.451 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:09.721+0000 7fcd50f5f700 1 -- 172.21.15.32:0/1737766658 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fcd4400b8c0 con 0x7fcd4c07ee10 2024-05-03T18:37:10.451 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:09.721+0000 7fcd50f5f700 1 -- 172.21.15.32:0/1737766658 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcd44004490 con 0x7fcd4c07ee10 2024-05-03T18:37:10.451 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:09.722+0000 7fcd541c5700 1 -- 172.21.15.32:0/1737766658 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd4c07ee10 msgr2=0x7fcd4c07d310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:37:10.451 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:09.722+0000 7fcd541c5700 1 --2- 172.21.15.32:0/1737766658 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd4c07ee10 0x7fcd4c07d310 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fcd44009e80 tx=0x7fcd4400b330 comp rx=0 tx=0).stop 2024-05-03T18:37:10.451 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:09.722+0000 7fcd541c5700 1 -- 172.21.15.32:0/1737766658 shutdown_connections 2024-05-03T18:37:10.451 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:09.722+0000 7fcd541c5700 1 --2- 172.21.15.32:0/1737766658 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd4c07ee10 0x7fcd4c07d310 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:10.451 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:09.722+0000 7fcd541c5700 1 -- 172.21.15.32:0/1737766658 >> 172.21.15.32:0/1737766658 conn(0x7fcd4c104650 msgr2=0x7fcd4c106a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:37:10.451 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:09.722+0000 7fcd541c5700 1 -- 172.21.15.32:0/1737766658 shutdown_connections 2024-05-03T18:37:10.451 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:09.722+0000 7fcd541c5700 1 -- 172.21.15.32:0/1737766658 wait complete. 2024-05-03T18:37:10.451 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:09.723+0000 7fcd541c5700 1 Processor -- start 2024-05-03T18:37:10.451 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:09.723+0000 7fcd541c5700 1 -- start start 2024-05-03T18:37:10.451 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:09.723+0000 7fcd541c5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd4c07ee10 0x7fcd4c19f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:10.451 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:09.723+0000 7fcd541c5700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fcd4c10f560 con 0x7fcd4c07ee10 2024-05-03T18:37:10.451 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:09.723+0000 7fcd51f61700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd4c07ee10 0x7fcd4c19f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:10.452 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:09.723+0000 7fcd51f61700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd4c07ee10 0x7fcd4c19f0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52532/0 (socket says 172.21.15.32:52532) 2024-05-03T18:37:10.452 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:09.723+0000 7fcd51f61700 1 -- 172.21.15.32:0/4261131945 learned_addr learned my addr 172.21.15.32:0/4261131945 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:37:10.452 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:09.723+0000 7fcd51f61700 1 -- 172.21.15.32:0/4261131945 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd4c19f5f0 con 0x7fcd4c07ee10 2024-05-03T18:37:10.452 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:09.723+0000 7fcd51f61700 1 --2- 172.21.15.32:0/4261131945 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd4c07ee10 0x7fcd4c19f0b0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fcd4400b7b0 tx=0x7fcd4400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:37:10.452 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:09.724+0000 7fcd4affd700 1 -- 172.21.15.32:0/4261131945 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcd44020020 con 0x7fcd4c07ee10 2024-05-03T18:37:10.452 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:09.724+0000 7fcd4affd700 1 -- 172.21.15.32:0/4261131945 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fcd44012430 con 0x7fcd4c07ee10 2024-05-03T18:37:10.452 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:09.724+0000 7fcd541c5700 1 -- 172.21.15.32:0/4261131945 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd4c19f880 con 0x7fcd4c07ee10 2024-05-03T18:37:10.452 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:09.724+0000 7fcd4affd700 1 -- 172.21.15.32:0/4261131945 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcd44017040 con 0x7fcd4c07ee10 2024-05-03T18:37:10.452 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:09.724+0000 7fcd541c5700 1 -- 172.21.15.32:0/4261131945 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd4c1a2570 con 0x7fcd4c07ee10 2024-05-03T18:37:10.452 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:09.725+0000 7fcd4affd700 1 -- 172.21.15.32:0/4261131945 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) ==== 50211+0+0 (secure 0 0 0) 0x7fcd44023020 con 0x7fcd4c07ee10 2024-05-03T18:37:10.452 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:09.725+0000 7fcd4affd700 1 --2- 172.21.15.32:0/4261131945 >> [v2:172.21.15.32:6800/3945405684,v1:172.21.15.32:6801/3945405684] conn(0x7fcd3403e420 0x7fcd340408e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:10.452 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:09.725+0000 7fcd541c5700 1 -- 172.21.15.32:0/4261131945 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd4c064b00 con 0x7fcd4c07ee10 2024-05-03T18:37:10.452 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:09.725+0000 7fcd51760700 1 --2- 172.21.15.32:0/4261131945 >> [v2:172.21.15.32:6800/3945405684,v1:172.21.15.32:6801/3945405684] conn(0x7fcd3403e420 0x7fcd340408e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:10.452 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:09.725+0000 7fcd4affd700 1 -- 172.21.15.32:0/4261131945 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fcd4400f050 con 0x7fcd4c07ee10 2024-05-03T18:37:10.453 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:09.725+0000 7fcd51760700 1 --2- 172.21.15.32:0/4261131945 >> [v2:172.21.15.32:6800/3945405684,v1:172.21.15.32:6801/3945405684] conn(0x7fcd3403e420 0x7fcd340408e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fcd40006fd0 tx=0x7fcd40006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:37:10.453 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:09.730+0000 7fcd4affd700 1 -- 172.21.15.32:0/4261131945 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fcd4402a020 con 0x7fcd4c07ee10 2024-05-03T18:37:10.453 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:09.942+0000 7fcd541c5700 1 -- 172.21.15.32:0/4261131945 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7fcd4c1a2740 con 0x7fcd4c07ee10 2024-05-03T18:37:10.453 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:09.943+0000 7fcd4affd700 1 -- 172.21.15.32:0/4261131945 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+436 (secure 0 0 0) 0x7fcd4c1a2740 con 0x7fcd4c07ee10 2024-05-03T18:37:10.453 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:09.945+0000 7fcd541c5700 1 -- 172.21.15.32:0/4261131945 >> [v2:172.21.15.32:6800/3945405684,v1:172.21.15.32:6801/3945405684] conn(0x7fcd3403e420 msgr2=0x7fcd340408e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:37:10.453 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:09.945+0000 7fcd541c5700 1 --2- 172.21.15.32:0/4261131945 >> [v2:172.21.15.32:6800/3945405684,v1:172.21.15.32:6801/3945405684] conn(0x7fcd3403e420 0x7fcd340408e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fcd40006fd0 tx=0x7fcd40006e40 comp rx=0 tx=0).stop 2024-05-03T18:37:10.453 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:09.945+0000 7fcd541c5700 1 -- 172.21.15.32:0/4261131945 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd4c07ee10 msgr2=0x7fcd4c19f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:37:10.453 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:09.945+0000 7fcd541c5700 1 --2- 172.21.15.32:0/4261131945 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd4c07ee10 0x7fcd4c19f0b0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fcd4400b7b0 tx=0x7fcd4400be20 comp rx=0 tx=0).stop 2024-05-03T18:37:10.453 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:09.946+0000 7fcd541c5700 1 -- 172.21.15.32:0/4261131945 shutdown_connections 2024-05-03T18:37:10.453 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:09.946+0000 7fcd541c5700 1 --2- 172.21.15.32:0/4261131945 >> [v2:172.21.15.32:6800/3945405684,v1:172.21.15.32:6801/3945405684] conn(0x7fcd3403e420 0x7fcd340408e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:10.453 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:09.946+0000 7fcd541c5700 1 --2- 172.21.15.32:0/4261131945 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd4c07ee10 0x7fcd4c19f0b0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:10.453 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:09.946+0000 7fcd541c5700 1 -- 172.21.15.32:0/4261131945 >> 172.21.15.32:0/4261131945 conn(0x7fcd4c104650 msgr2=0x7fcd4c1051b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:37:10.453 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:09.946+0000 7fcd541c5700 1 -- 172.21.15.32:0/4261131945 shutdown_connections 2024-05-03T18:37:10.453 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:09.946+0000 7fcd541c5700 1 -- 172.21.15.32:0/4261131945 wait complete. 2024-05-03T18:37:10.453 INFO:teuthology.orchestra.run.smithi032.stdout:Enabling cephadm module... 2024-05-03T18:37:11.546 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:37:11 smithi032 ceph-mon[29763]: from='client.? 172.21.15.32:0/712549075' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-05-03T18:37:11.547 INFO:journalctl@ceph.mgr.x.smithi032.stdout:May 03 18:37:11 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-x[29995]: ignoring --setuser ceph since I am not root 2024-05-03T18:37:11.547 INFO:journalctl@ceph.mgr.x.smithi032.stdout:May 03 18:37:11 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-x[29995]: ignoring --setgroup ceph since I am not root 2024-05-03T18:37:11.547 INFO:journalctl@ceph.mgr.x.smithi032.stdout:May 03 18:37:11 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-x[29995]: 2024-05-03T18:37:11.354+0000 7f283e3ae200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-03T18:37:11.547 INFO:journalctl@ceph.mgr.x.smithi032.stdout:May 03 18:37:11 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-x[29995]: 2024-05-03T18:37:11.546+0000 7f283e3ae200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-03T18:37:11.664 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:10.838+0000 7f9c88b7b700 1 Processor -- start 2024-05-03T18:37:11.665 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:10.838+0000 7f9c88b7b700 1 -- start start 2024-05-03T18:37:11.665 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:10.838+0000 7f9c88b7b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9c84107e90 0x7f9c841082b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:11.665 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:10.838+0000 7f9c88b7b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f9c841087f0 con 0x7f9c84107e90 2024-05-03T18:37:11.670 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:10.838+0000 7f9c8259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9c84107e90 0x7f9c841082b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:11.670 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:10.839+0000 7f9c8259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9c84107e90 0x7f9c841082b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52548/0 (socket says 172.21.15.32:52548) 2024-05-03T18:37:11.670 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:10.839+0000 7f9c8259c700 1 -- 172.21.15.32:0/1181899593 learned_addr learned my addr 172.21.15.32:0/1181899593 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:37:11.670 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:10.839+0000 7f9c8259c700 1 -- 172.21.15.32:0/1181899593 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c84108ff0 con 0x7f9c84107e90 2024-05-03T18:37:11.670 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:10.839+0000 7f9c8259c700 1 --2- 172.21.15.32:0/1181899593 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9c84107e90 0x7f9c841082b0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f9c740098a0 tx=0x7f9c74009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7881b8cc3711d478 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:37:11.671 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:10.839+0000 7f9c8159a700 1 -- 172.21.15.32:0/1181899593 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9c74004160 con 0x7f9c84107e90 2024-05-03T18:37:11.671 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:10.839+0000 7f9c8159a700 1 -- 172.21.15.32:0/1181899593 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f9c74004300 con 0x7f9c84107e90 2024-05-03T18:37:11.671 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:10.839+0000 7f9c8159a700 1 -- 172.21.15.32:0/1181899593 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9c74011af0 con 0x7f9c84107e90 2024-05-03T18:37:11.671 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:10.840+0000 7f9c88b7b700 1 -- 172.21.15.32:0/1181899593 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9c84107e90 msgr2=0x7f9c841082b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:37:11.671 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:10.840+0000 7f9c88b7b700 1 --2- 172.21.15.32:0/1181899593 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9c84107e90 0x7f9c841082b0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f9c740098a0 tx=0x7f9c74009bb0 comp rx=0 tx=0).stop 2024-05-03T18:37:11.671 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:10.840+0000 7f9c88b7b700 1 -- 172.21.15.32:0/1181899593 shutdown_connections 2024-05-03T18:37:11.671 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:10.840+0000 7f9c88b7b700 1 --2- 172.21.15.32:0/1181899593 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9c84107e90 0x7f9c841082b0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:11.671 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:10.840+0000 7f9c88b7b700 1 -- 172.21.15.32:0/1181899593 >> 172.21.15.32:0/1181899593 conn(0x7f9c84103450 msgr2=0x7f9c84105870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:37:11.671 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:10.840+0000 7f9c88b7b700 1 -- 172.21.15.32:0/1181899593 shutdown_connections 2024-05-03T18:37:11.671 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:10.840+0000 7f9c88b7b700 1 -- 172.21.15.32:0/1181899593 wait complete. 2024-05-03T18:37:11.671 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:10.841+0000 7f9c88b7b700 1 Processor -- start 2024-05-03T18:37:11.671 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:10.841+0000 7f9c88b7b700 1 -- start start 2024-05-03T18:37:11.671 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:10.841+0000 7f9c88b7b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9c84107e90 0x7f9c8407ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:11.671 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:10.841+0000 7f9c88b7b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f9c8410c430 con 0x7f9c84107e90 2024-05-03T18:37:11.672 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:10.841+0000 7f9c8259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9c84107e90 0x7f9c8407ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:11.672 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:10.841+0000 7f9c8259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9c84107e90 0x7f9c8407ca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52564/0 (socket says 172.21.15.32:52564) 2024-05-03T18:37:11.672 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:10.841+0000 7f9c8259c700 1 -- 172.21.15.32:0/712549075 learned_addr learned my addr 172.21.15.32:0/712549075 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:37:11.672 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:10.841+0000 7f9c8259c700 1 -- 172.21.15.32:0/712549075 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c8419fca0 con 0x7f9c84107e90 2024-05-03T18:37:11.672 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:10.841+0000 7f9c8259c700 1 --2- 172.21.15.32:0/712549075 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9c84107e90 0x7f9c8407ca60 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f9c74009fb0 tx=0x7f9c74011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:37:11.672 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:10.841+0000 7f9c737fe700 1 -- 172.21.15.32:0/712549075 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9c740036a0 con 0x7f9c84107e90 2024-05-03T18:37:11.672 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:10.841+0000 7f9c737fe700 1 -- 172.21.15.32:0/712549075 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f9c7401a430 con 0x7f9c84107e90 2024-05-03T18:37:11.672 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:10.841+0000 7f9c88b7b700 1 -- 172.21.15.32:0/712549075 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c8419fe70 con 0x7f9c84107e90 2024-05-03T18:37:11.672 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:10.842+0000 7f9c737fe700 1 -- 172.21.15.32:0/712549075 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9c74019550 con 0x7f9c84107e90 2024-05-03T18:37:11.672 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:10.842+0000 7f9c88b7b700 1 -- 172.21.15.32:0/712549075 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c841a0210 con 0x7f9c84107e90 2024-05-03T18:37:11.672 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:10.842+0000 7f9c737fe700 1 -- 172.21.15.32:0/712549075 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) ==== 50211+0+0 (secure 0 0 0) 0x7f9c7401a8e0 con 0x7f9c84107e90 2024-05-03T18:37:11.672 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:10.843+0000 7f9c88b7b700 1 -- 172.21.15.32:0/712549075 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c84064b00 con 0x7f9c84107e90 2024-05-03T18:37:11.672 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:10.843+0000 7f9c737fe700 1 --2- 172.21.15.32:0/712549075 >> [v2:172.21.15.32:6800/3945405684,v1:172.21.15.32:6801/3945405684] conn(0x7f9c6c03e420 0x7f9c6c0408e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:11.672 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:10.843+0000 7f9c737fe700 1 -- 172.21.15.32:0/712549075 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f9c74053790 con 0x7f9c84107e90 2024-05-03T18:37:11.673 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:10.843+0000 7f9c81d9b700 1 --2- 172.21.15.32:0/712549075 >> [v2:172.21.15.32:6800/3945405684,v1:172.21.15.32:6801/3945405684] conn(0x7f9c6c03e420 0x7f9c6c0408e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:11.673 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:10.843+0000 7f9c81d9b700 1 --2- 172.21.15.32:0/712549075 >> [v2:172.21.15.32:6800/3945405684,v1:172.21.15.32:6801/3945405684] conn(0x7f9c6c03e420 0x7f9c6c0408e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9c78006fd0 tx=0x7f9c78006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:37:11.673 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:10.848+0000 7f9c737fe700 1 -- 172.21.15.32:0/712549075 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f9c740531b0 con 0x7f9c84107e90 2024-05-03T18:37:11.673 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:11.126+0000 7f9c88b7b700 1 -- 172.21.15.32:0/712549075 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f9c841a0620 con 0x7f9c84107e90 2024-05-03T18:37:11.673 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:11.177+0000 7f9c737fe700 1 -- 172.21.15.32:0/712549075 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f9c841a0620 con 0x7f9c84107e90 2024-05-03T18:37:11.673 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:11.179+0000 7f9c737fe700 1 -- 172.21.15.32:0/712549075 <== mon.0 v2:172.21.15.32:3300/0 8 ==== mgrmap(e 5) ==== 50222+0+0 (secure 0 0 0) 0x7f9c74018530 con 0x7f9c84107e90 2024-05-03T18:37:11.673 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:11.180+0000 7f9c88b7b700 1 -- 172.21.15.32:0/712549075 >> [v2:172.21.15.32:6800/3945405684,v1:172.21.15.32:6801/3945405684] conn(0x7f9c6c03e420 msgr2=0x7f9c6c0408e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:37:11.673 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:11.180+0000 7f9c88b7b700 1 --2- 172.21.15.32:0/712549075 >> [v2:172.21.15.32:6800/3945405684,v1:172.21.15.32:6801/3945405684] conn(0x7f9c6c03e420 0x7f9c6c0408e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9c78006fd0 tx=0x7f9c78006e40 comp rx=0 tx=0).stop 2024-05-03T18:37:11.673 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:11.180+0000 7f9c88b7b700 1 -- 172.21.15.32:0/712549075 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9c84107e90 msgr2=0x7f9c8407ca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:37:11.673 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:11.180+0000 7f9c88b7b700 1 --2- 172.21.15.32:0/712549075 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9c84107e90 0x7f9c8407ca60 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f9c74009fb0 tx=0x7f9c74011940 comp rx=0 tx=0).stop 2024-05-03T18:37:11.673 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:11.180+0000 7f9c88b7b700 1 -- 172.21.15.32:0/712549075 shutdown_connections 2024-05-03T18:37:11.673 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:11.180+0000 7f9c88b7b700 1 --2- 172.21.15.32:0/712549075 >> [v2:172.21.15.32:6800/3945405684,v1:172.21.15.32:6801/3945405684] conn(0x7f9c6c03e420 0x7f9c6c0408e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:11.673 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:11.180+0000 7f9c88b7b700 1 --2- 172.21.15.32:0/712549075 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9c84107e90 0x7f9c8407ca60 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:11.673 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:11.180+0000 7f9c88b7b700 1 -- 172.21.15.32:0/712549075 >> 172.21.15.32:0/712549075 conn(0x7f9c84103450 msgr2=0x7f9c84103fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:37:11.673 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:11.180+0000 7f9c88b7b700 1 -- 172.21.15.32:0/712549075 shutdown_connections 2024-05-03T18:37:11.674 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:11.180+0000 7f9c88b7b700 1 -- 172.21.15.32:0/712549075 wait complete. 2024-05-03T18:37:11.817 INFO:journalctl@ceph.mgr.x.smithi032.stdout:May 03 18:37:11 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-x[29995]: 2024-05-03T18:37:11.653+0000 7f283e3ae200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-03T18:37:12.317 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:37:12 smithi032 ceph-mon[29763]: from='client.? 172.21.15.32:0/712549075' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-05-03T18:37:12.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:37:12 smithi032 ceph-mon[29763]: mgrmap e5: x(active, since 13s) 2024-05-03T18:37:12.318 INFO:journalctl@ceph.mgr.x.smithi032.stdout:May 03 18:37:11 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-x[29995]: 2024-05-03T18:37:11.980+0000 7f283e3ae200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-03T18:37:12.318 INFO:journalctl@ceph.mgr.x.smithi032.stdout:May 03 18:37:12 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-x[29995]: 2024-05-03T18:37:12.067+0000 7f283e3ae200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-03T18:37:13.317 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:37:13 smithi032 ceph-mon[29763]: from='client.? 172.21.15.32:0/3412998082' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-05-03T18:37:13.635 INFO:journalctl@ceph.mgr.x.smithi032.stdout:May 03 18:37:13 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-x[29995]: 2024-05-03T18:37:13.318+0000 7f283e3ae200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-03T18:37:13.635 INFO:journalctl@ceph.mgr.x.smithi032.stdout:May 03 18:37:13 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-x[29995]: 2024-05-03T18:37:13.494+0000 7f283e3ae200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-03T18:37:13.644 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout { 2024-05-03T18:37:13.644 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-05-03T18:37:13.644 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-03T18:37:13.644 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-05-03T18:37:13.644 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-05-03T18:37:13.644 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout } 2024-05-03T18:37:13.644 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:12.791+0000 7f70af0dc700 1 Processor -- start 2024-05-03T18:37:13.644 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:12.791+0000 7f70af0dc700 1 -- start start 2024-05-03T18:37:13.644 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:12.791+0000 7f70af0dc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70a8109010 0x7f70a8109430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:13.644 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:12.791+0000 7f70af0dc700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f70a8109a00 con 0x7f70a8109010 2024-05-03T18:37:13.644 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:12.791+0000 7f70ace78700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70a8109010 0x7f70a8109430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:13.644 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:12.791+0000 7f70ace78700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70a8109010 0x7f70a8109430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52596/0 (socket says 172.21.15.32:52596) 2024-05-03T18:37:13.644 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:12.791+0000 7f70ace78700 1 -- 172.21.15.32:0/603177164 learned_addr learned my addr 172.21.15.32:0/603177164 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:37:13.644 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:12.792+0000 7f70ace78700 1 -- 172.21.15.32:0/603177164 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70a810a170 con 0x7f70a8109010 2024-05-03T18:37:13.645 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:12.792+0000 7f70ace78700 1 --2- 172.21.15.32:0/603177164 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70a8109010 0x7f70a8109430 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f709c0098a0 tx=0x7f709c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bae31c533d27b787 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:37:13.645 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:12.792+0000 7f70a77fe700 1 -- 172.21.15.32:0/603177164 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f709c004160 con 0x7f70a8109010 2024-05-03T18:37:13.645 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:12.792+0000 7f70a77fe700 1 -- 172.21.15.32:0/603177164 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f709c004300 con 0x7f70a8109010 2024-05-03T18:37:13.645 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:12.792+0000 7f70a77fe700 1 -- 172.21.15.32:0/603177164 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f709c011af0 con 0x7f70a8109010 2024-05-03T18:37:13.645 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:12.793+0000 7f70af0dc700 1 -- 172.21.15.32:0/603177164 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70a8109010 msgr2=0x7f70a8109430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:37:13.645 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:12.793+0000 7f70af0dc700 1 --2- 172.21.15.32:0/603177164 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70a8109010 0x7f70a8109430 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f709c0098a0 tx=0x7f709c009bb0 comp rx=0 tx=0).stop 2024-05-03T18:37:13.645 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:12.793+0000 7f70af0dc700 1 -- 172.21.15.32:0/603177164 shutdown_connections 2024-05-03T18:37:13.645 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:12.793+0000 7f70af0dc700 1 --2- 172.21.15.32:0/603177164 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70a8109010 0x7f70a8109430 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:13.645 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:12.793+0000 7f70af0dc700 1 -- 172.21.15.32:0/603177164 >> 172.21.15.32:0/603177164 conn(0x7f70a8104650 msgr2=0x7f70a8106a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:37:13.645 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:12.793+0000 7f70af0dc700 1 -- 172.21.15.32:0/603177164 shutdown_connections 2024-05-03T18:37:13.645 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:12.793+0000 7f70af0dc700 1 -- 172.21.15.32:0/603177164 wait complete. 2024-05-03T18:37:13.645 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:12.793+0000 7f70af0dc700 1 Processor -- start 2024-05-03T18:37:13.645 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:12.793+0000 7f70af0dc700 1 -- start start 2024-05-03T18:37:13.646 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:12.794+0000 7f70af0dc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70a8109010 0x7f70a819cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:13.646 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:12.794+0000 7f70af0dc700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f70a810d520 con 0x7f70a8109010 2024-05-03T18:37:13.646 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:12.794+0000 7f70ace78700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70a8109010 0x7f70a819cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:13.646 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:12.794+0000 7f70ace78700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70a8109010 0x7f70a819cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52606/0 (socket says 172.21.15.32:52606) 2024-05-03T18:37:13.646 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:12.794+0000 7f70ace78700 1 -- 172.21.15.32:0/3412998082 learned_addr learned my addr 172.21.15.32:0/3412998082 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:37:13.646 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:12.794+0000 7f70ace78700 1 -- 172.21.15.32:0/3412998082 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70a819d480 con 0x7f70a8109010 2024-05-03T18:37:13.646 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:12.794+0000 7f70ace78700 1 --2- 172.21.15.32:0/3412998082 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70a8109010 0x7f70a819cf40 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f709c0117b0 tx=0x7f709c011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:37:13.646 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:12.794+0000 7f70a5ffb700 1 -- 172.21.15.32:0/3412998082 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f709c003af0 con 0x7f70a8109010 2024-05-03T18:37:13.646 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:12.794+0000 7f70a5ffb700 1 -- 172.21.15.32:0/3412998082 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f709c020070 con 0x7f70a8109010 2024-05-03T18:37:13.646 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:12.794+0000 7f70af0dc700 1 -- 172.21.15.32:0/3412998082 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70a819d710 con 0x7f70a8109010 2024-05-03T18:37:13.646 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:12.794+0000 7f70a5ffb700 1 -- 172.21.15.32:0/3412998082 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f709c019630 con 0x7f70a8109010 2024-05-03T18:37:13.646 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:12.795+0000 7f70af0dc700 1 -- 172.21.15.32:0/3412998082 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70a81a0400 con 0x7f70a8109010 2024-05-03T18:37:13.646 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:12.795+0000 7f70af0dc700 1 -- 172.21.15.32:0/3412998082 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70a8064b00 con 0x7f70a8109010 2024-05-03T18:37:13.646 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:12.796+0000 7f70a5ffb700 1 -- 172.21.15.32:0/3412998082 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 50222+0+0 (secure 0 0 0) 0x7f709c01a9c0 con 0x7f70a8109010 2024-05-03T18:37:13.647 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:12.796+0000 7f70a5ffb700 1 --2- 172.21.15.32:0/3412998082 >> [v2:172.21.15.32:6800/3945405684,v1:172.21.15.32:6801/3945405684] conn(0x7f709803e4c0 0x7f7098040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:13.647 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:12.796+0000 7f70a5ffb700 1 -- 172.21.15.32:0/3412998082 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f709c053aa0 con 0x7f70a8109010 2024-05-03T18:37:13.647 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:12.796+0000 7f70a7fff700 1 -- 172.21.15.32:0/3412998082 >> [v2:172.21.15.32:6800/3945405684,v1:172.21.15.32:6801/3945405684] conn(0x7f709803e4c0 msgr2=0x7f7098040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6800/3945405684 2024-05-03T18:37:13.647 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:12.796+0000 7f70a7fff700 1 --2- 172.21.15.32:0/3412998082 >> [v2:172.21.15.32:6800/3945405684,v1:172.21.15.32:6801/3945405684] conn(0x7f709803e4c0 0x7f7098040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-03T18:37:13.647 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:12.801+0000 7f70a5ffb700 1 -- 172.21.15.32:0/3412998082 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f709c0172b0 con 0x7f70a8109010 2024-05-03T18:37:13.647 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:12.997+0000 7f70a7fff700 1 -- 172.21.15.32:0/3412998082 >> [v2:172.21.15.32:6800/3945405684,v1:172.21.15.32:6801/3945405684] conn(0x7f709803e4c0 msgr2=0x7f7098040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6800/3945405684 2024-05-03T18:37:13.647 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:12.997+0000 7f70a7fff700 1 --2- 172.21.15.32:0/3412998082 >> [v2:172.21.15.32:6800/3945405684,v1:172.21.15.32:6801/3945405684] conn(0x7f709803e4c0 0x7f7098040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-03T18:37:13.648 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:13.075+0000 7f70af0dc700 1 -- 172.21.15.32:0/3412998082 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f70a81a05d0 con 0x7f70a8109010 2024-05-03T18:37:13.648 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:13.076+0000 7f70a5ffb700 1 -- 172.21.15.32:0/3412998082 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7f70a81a05d0 con 0x7f70a8109010 2024-05-03T18:37:13.648 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:13.078+0000 7f70af0dc700 1 -- 172.21.15.32:0/3412998082 >> [v2:172.21.15.32:6800/3945405684,v1:172.21.15.32:6801/3945405684] conn(0x7f709803e4c0 msgr2=0x7f7098040980 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-03T18:37:13.648 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:13.078+0000 7f70af0dc700 1 --2- 172.21.15.32:0/3412998082 >> [v2:172.21.15.32:6800/3945405684,v1:172.21.15.32:6801/3945405684] conn(0x7f709803e4c0 0x7f7098040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:13.648 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:13.078+0000 7f70af0dc700 1 -- 172.21.15.32:0/3412998082 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70a8109010 msgr2=0x7f70a819cf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:37:13.648 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:13.078+0000 7f70af0dc700 1 --2- 172.21.15.32:0/3412998082 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70a8109010 0x7f70a819cf40 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f709c0117b0 tx=0x7f709c011b70 comp rx=0 tx=0).stop 2024-05-03T18:37:13.648 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:13.078+0000 7f70af0dc700 1 -- 172.21.15.32:0/3412998082 shutdown_connections 2024-05-03T18:37:13.648 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:13.078+0000 7f70af0dc700 1 --2- 172.21.15.32:0/3412998082 >> [v2:172.21.15.32:6800/3945405684,v1:172.21.15.32:6801/3945405684] conn(0x7f709803e4c0 0x7f7098040980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:13.648 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:13.078+0000 7f70af0dc700 1 --2- 172.21.15.32:0/3412998082 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70a8109010 0x7f70a819cf40 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:13.648 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:13.078+0000 7f70af0dc700 1 -- 172.21.15.32:0/3412998082 >> 172.21.15.32:0/3412998082 conn(0x7f70a8104650 msgr2=0x7f70a81051e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:37:13.648 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:13.078+0000 7f70af0dc700 1 -- 172.21.15.32:0/3412998082 shutdown_connections 2024-05-03T18:37:13.648 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:13.078+0000 7f70af0dc700 1 -- 172.21.15.32:0/3412998082 wait complete. 2024-05-03T18:37:13.648 INFO:teuthology.orchestra.run.smithi032.stdout:Waiting for the mgr to restart... 2024-05-03T18:37:13.649 INFO:teuthology.orchestra.run.smithi032.stdout:Waiting for mgr epoch 5... 2024-05-03T18:37:13.962 INFO:journalctl@ceph.mgr.x.smithi032.stdout:May 03 18:37:13 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-x[29995]: 2024-05-03T18:37:13.675+0000 7f283e3ae200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-03T18:37:14.290 INFO:journalctl@ceph.mgr.x.smithi032.stdout:May 03 18:37:13 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-x[29995]: 2024-05-03T18:37:13.980+0000 7f283e3ae200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-03T18:37:14.290 INFO:journalctl@ceph.mgr.x.smithi032.stdout:May 03 18:37:14 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-x[29995]: 2024-05-03T18:37:14.078+0000 7f283e3ae200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-03T18:37:14.290 INFO:journalctl@ceph.mgr.x.smithi032.stdout:May 03 18:37:14 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-x[29995]: 2024-05-03T18:37:14.289+0000 7f283e3ae200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-03T18:37:14.559 INFO:journalctl@ceph.mgr.x.smithi032.stdout:May 03 18:37:14 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-x[29995]: 2024-05-03T18:37:14.488+0000 7f283e3ae200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-03T18:37:15.498 INFO:journalctl@ceph.mgr.x.smithi032.stdout:May 03 18:37:15 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-x[29995]: 2024-05-03T18:37:15.116+0000 7f283e3ae200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-03T18:37:15.498 INFO:journalctl@ceph.mgr.x.smithi032.stdout:May 03 18:37:15 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-x[29995]: 2024-05-03T18:37:15.343+0000 7f283e3ae200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-03T18:37:16.068 INFO:journalctl@ceph.mgr.x.smithi032.stdout:May 03 18:37:15 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-x[29995]: 2024-05-03T18:37:15.779+0000 7f283e3ae200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-03T18:37:16.068 INFO:journalctl@ceph.mgr.x.smithi032.stdout:May 03 18:37:15 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-x[29995]: 2024-05-03T18:37:15.878+0000 7f283e3ae200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-03T18:37:16.567 INFO:journalctl@ceph.mgr.x.smithi032.stdout:May 03 18:37:16 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-x[29995]: 2024-05-03T18:37:16.261+0000 7f283e3ae200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-03T18:37:16.977 INFO:journalctl@ceph.mgr.x.smithi032.stdout:May 03 18:37:16 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-x[29995]: 2024-05-03T18:37:16.630+0000 7f283e3ae200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-03T18:37:16.977 INFO:journalctl@ceph.mgr.x.smithi032.stdout:May 03 18:37:16 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-x[29995]: 2024-05-03T18:37:16.725+0000 7f283e3ae200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-03T18:37:17.318 INFO:journalctl@ceph.mgr.x.smithi032.stdout:May 03 18:37:16 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-x[29995]: 2024-05-03T18:37:16.976+0000 7f283e3ae200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-03T18:37:17.318 INFO:journalctl@ceph.mgr.x.smithi032.stdout:May 03 18:37:17 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-x[29995]: 2024-05-03T18:37:17.113+0000 7f283e3ae200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-03T18:37:17.817 INFO:journalctl@ceph.mgr.x.smithi032.stdout:May 03 18:37:17 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-x[29995]: 2024-05-03T18:37:17.428+0000 7f283e3ae200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-03T18:37:17.818 INFO:journalctl@ceph.mgr.x.smithi032.stdout:May 03 18:37:17 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-x[29995]: 2024-05-03T18:37:17.547+0000 7f283e3ae200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-03T18:37:17.818 INFO:journalctl@ceph.mgr.x.smithi032.stdout:May 03 18:37:17 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-x[29995]: 2024-05-03T18:37:17.642+0000 7f283e3ae200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-03T18:37:18.543 INFO:journalctl@ceph.mgr.x.smithi032.stdout:May 03 18:37:18 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-x[29995]: 2024-05-03T18:37:18.264+0000 7f283e3ae200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-03T18:37:18.543 INFO:journalctl@ceph.mgr.x.smithi032.stdout:May 03 18:37:18 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-x[29995]: 2024-05-03T18:37:18.489+0000 7f283e3ae200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-03T18:37:18.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:37:18 smithi032 ceph-mon[29763]: Active manager daemon x restarted 2024-05-03T18:37:18.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:37:18 smithi032 ceph-mon[29763]: Activating manager daemon x 2024-05-03T18:37:18.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:37:18 smithi032 ceph-mon[29763]: osdmap e2: 0 total, 0 up, 0 in 2024-05-03T18:37:18.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:37:18 smithi032 ceph-mon[29763]: mgrmap e6: x(active, starting, since 0.00357741s) 2024-05-03T18:37:18.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:37:18 smithi032 ceph-mon[29763]: from='mgr.14120 172.21.15.32:0/2820864439' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-03T18:37:18.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:37:18 smithi032 ceph-mon[29763]: from='mgr.14120 172.21.15.32:0/2820864439' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-03T18:37:18.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:37:18 smithi032 ceph-mon[29763]: from='mgr.14120 172.21.15.32:0/2820864439' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-03T18:37:18.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:37:18 smithi032 ceph-mon[29763]: from='mgr.14120 172.21.15.32:0/2820864439' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-03T18:37:18.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:37:18 smithi032 ceph-mon[29763]: from='mgr.14120 172.21.15.32:0/2820864439' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-03T18:37:18.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:37:18 smithi032 ceph-mon[29763]: Manager daemon x is now available 2024-05-03T18:37:18.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:37:18 smithi032 ceph-mon[29763]: from='mgr.14120 172.21.15.32:0/2820864439' entity='mgr.x' 2024-05-03T18:37:18.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:37:18 smithi032 ceph-mon[29763]: from='mgr.14120 172.21.15.32:0/2820864439' entity='mgr.x' 2024-05-03T18:37:18.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:37:18 smithi032 ceph-mon[29763]: from='mgr.14120 172.21.15.32:0/2820864439' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:37:18.819 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:37:18 smithi032 ceph-mon[29763]: from='mgr.14120 172.21.15.32:0/2820864439' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:37:18.819 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:37:18 smithi032 ceph-mon[29763]: from='mgr.14120 172.21.15.32:0/2820864439' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-03T18:37:19.807 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:37:19 smithi032 ceph-mon[29763]: Found migration_current of "None". Setting to last migration. 2024-05-03T18:37:19.808 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:37:19 smithi032 ceph-mon[29763]: from='mgr.14120 172.21.15.32:0/2820864439' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-03T18:37:19.808 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:37:19 smithi032 ceph-mon[29763]: from='mgr.14120 172.21.15.32:0/2820864439' entity='mgr.x' 2024-05-03T18:37:19.808 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:37:19 smithi032 ceph-mon[29763]: from='mgr.14120 172.21.15.32:0/2820864439' entity='mgr.x' 2024-05-03T18:37:19.808 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:37:19 smithi032 ceph-mon[29763]: mgrmap e7: x(active, since 1.00565s) 2024-05-03T18:37:20.068 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout { 2024-05-03T18:37:20.068 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-05-03T18:37:20.068 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "initialized": true 2024-05-03T18:37:20.068 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout } 2024-05-03T18:37:20.068 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:14.090+0000 7fb8972f3700 1 Processor -- start 2024-05-03T18:37:20.069 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:14.090+0000 7fb8972f3700 1 -- start start 2024-05-03T18:37:20.069 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:14.090+0000 7fb8972f3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb890107830 0x7fb890107c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:20.069 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:14.090+0000 7fb8972f3700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fb890076d60 con 0x7fb890107830 2024-05-03T18:37:20.069 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:14.091+0000 7fb89508f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb890107830 0x7fb890107c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:20.069 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:14.091+0000 7fb89508f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb890107830 0x7fb890107c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35464/0 (socket says 172.21.15.32:35464) 2024-05-03T18:37:20.069 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:14.091+0000 7fb89508f700 1 -- 172.21.15.32:0/1551252651 learned_addr learned my addr 172.21.15.32:0/1551252651 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:37:20.069 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:14.091+0000 7fb89508f700 1 -- 172.21.15.32:0/1551252651 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb890108190 con 0x7fb890107830 2024-05-03T18:37:20.069 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:14.091+0000 7fb89508f700 1 --2- 172.21.15.32:0/1551252651 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb890107830 0x7fb890107c50 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fb8800098a0 tx=0x7fb880009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=beb139ea4fe08e14 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:37:20.069 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:14.091+0000 7fb887fff700 1 -- 172.21.15.32:0/1551252651 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb880004160 con 0x7fb890107830 2024-05-03T18:37:20.069 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:14.091+0000 7fb887fff700 1 -- 172.21.15.32:0/1551252651 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fb880004300 con 0x7fb890107830 2024-05-03T18:37:20.069 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:14.091+0000 7fb887fff700 1 -- 172.21.15.32:0/1551252651 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb880011af0 con 0x7fb890107830 2024-05-03T18:37:20.069 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:14.092+0000 7fb8972f3700 1 -- 172.21.15.32:0/1551252651 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb890107830 msgr2=0x7fb890107c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:37:20.069 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:14.092+0000 7fb8972f3700 1 --2- 172.21.15.32:0/1551252651 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb890107830 0x7fb890107c50 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fb8800098a0 tx=0x7fb880009bb0 comp rx=0 tx=0).stop 2024-05-03T18:37:20.070 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:14.092+0000 7fb8972f3700 1 -- 172.21.15.32:0/1551252651 shutdown_connections 2024-05-03T18:37:20.070 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:14.092+0000 7fb8972f3700 1 --2- 172.21.15.32:0/1551252651 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb890107830 0x7fb890107c50 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:20.070 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:14.092+0000 7fb8972f3700 1 -- 172.21.15.32:0/1551252651 >> 172.21.15.32:0/1551252651 conn(0x7fb8901034d0 msgr2=0x7fb8901058f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:37:20.070 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:14.092+0000 7fb8972f3700 1 -- 172.21.15.32:0/1551252651 shutdown_connections 2024-05-03T18:37:20.070 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:14.092+0000 7fb8972f3700 1 -- 172.21.15.32:0/1551252651 wait complete. 2024-05-03T18:37:20.070 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:14.093+0000 7fb8972f3700 1 Processor -- start 2024-05-03T18:37:20.070 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:14.093+0000 7fb8972f3700 1 -- start start 2024-05-03T18:37:20.070 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:14.093+0000 7fb8972f3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb890107830 0x7fb89019f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:20.070 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:14.093+0000 7fb8972f3700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fb8901091c0 con 0x7fb890107830 2024-05-03T18:37:20.070 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:14.093+0000 7fb89508f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb890107830 0x7fb89019f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:20.070 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:14.093+0000 7fb89508f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb890107830 0x7fb89019f0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35468/0 (socket says 172.21.15.32:35468) 2024-05-03T18:37:20.070 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:14.093+0000 7fb89508f700 1 -- 172.21.15.32:0/3406510207 learned_addr learned my addr 172.21.15.32:0/3406510207 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:37:20.070 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:14.093+0000 7fb89508f700 1 -- 172.21.15.32:0/3406510207 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb89019f610 con 0x7fb890107830 2024-05-03T18:37:20.070 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:14.093+0000 7fb89508f700 1 --2- 172.21.15.32:0/3406510207 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb890107830 0x7fb89019f0d0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fb880003f40 tx=0x7fb880011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:37:20.070 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:14.094+0000 7fb8867fc700 1 -- 172.21.15.32:0/3406510207 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb8800038b0 con 0x7fb890107830 2024-05-03T18:37:20.071 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:14.094+0000 7fb8867fc700 1 -- 172.21.15.32:0/3406510207 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fb880003a50 con 0x7fb890107830 2024-05-03T18:37:20.071 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:14.094+0000 7fb8867fc700 1 -- 172.21.15.32:0/3406510207 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb88001a5c0 con 0x7fb890107830 2024-05-03T18:37:20.071 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:14.094+0000 7fb8972f3700 1 -- 172.21.15.32:0/3406510207 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb89019f8a0 con 0x7fb890107830 2024-05-03T18:37:20.071 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:14.094+0000 7fb8972f3700 1 -- 172.21.15.32:0/3406510207 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8901a2590 con 0x7fb890107830 2024-05-03T18:37:20.071 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:14.095+0000 7fb8867fc700 1 -- 172.21.15.32:0/3406510207 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) ==== 50222+0+0 (secure 0 0 0) 0x7fb880024430 con 0x7fb890107830 2024-05-03T18:37:20.071 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:14.095+0000 7fb8867fc700 1 --2- 172.21.15.32:0/3406510207 >> [v2:172.21.15.32:6800/3945405684,v1:172.21.15.32:6801/3945405684] conn(0x7fb87c03e420 0x7fb87c0408e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:20.071 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:14.095+0000 7fb8867fc700 1 -- 172.21.15.32:0/3406510207 --> [v2:172.21.15.32:6800/3945405684,v1:172.21.15.32:6801/3945405684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fb87c040ff0 con 0x7fb87c03e420 2024-05-03T18:37:20.071 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:14.095+0000 7fb8867fc700 1 -- 172.21.15.32:0/3406510207 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fb880053a50 con 0x7fb890107830 2024-05-03T18:37:20.071 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:14.095+0000 7fb89488e700 1 -- 172.21.15.32:0/3406510207 >> [v2:172.21.15.32:6800/3945405684,v1:172.21.15.32:6801/3945405684] conn(0x7fb87c03e420 msgr2=0x7fb87c0408e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6800/3945405684 2024-05-03T18:37:20.071 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:14.095+0000 7fb89488e700 1 --2- 172.21.15.32:0/3406510207 >> [v2:172.21.15.32:6800/3945405684,v1:172.21.15.32:6801/3945405684] conn(0x7fb87c03e420 0x7fb87c0408e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-03T18:37:20.071 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:14.296+0000 7fb89488e700 1 -- 172.21.15.32:0/3406510207 >> [v2:172.21.15.32:6800/3945405684,v1:172.21.15.32:6801/3945405684] conn(0x7fb87c03e420 msgr2=0x7fb87c0408e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6800/3945405684 2024-05-03T18:37:20.071 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:14.296+0000 7fb89488e700 1 --2- 172.21.15.32:0/3406510207 >> [v2:172.21.15.32:6800/3945405684,v1:172.21.15.32:6801/3945405684] conn(0x7fb87c03e420 0x7fb87c0408e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-03T18:37:20.071 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:14.697+0000 7fb89488e700 1 -- 172.21.15.32:0/3406510207 >> [v2:172.21.15.32:6800/3945405684,v1:172.21.15.32:6801/3945405684] conn(0x7fb87c03e420 msgr2=0x7fb87c0408e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6800/3945405684 2024-05-03T18:37:20.071 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:14.697+0000 7fb89488e700 1 --2- 172.21.15.32:0/3406510207 >> [v2:172.21.15.32:6800/3945405684,v1:172.21.15.32:6801/3945405684] conn(0x7fb87c03e420 0x7fb87c0408e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-03T18:37:20.072 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:15.498+0000 7fb89488e700 1 -- 172.21.15.32:0/3406510207 >> [v2:172.21.15.32:6800/3945405684,v1:172.21.15.32:6801/3945405684] conn(0x7fb87c03e420 msgr2=0x7fb87c0408e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6800/3945405684 2024-05-03T18:37:20.072 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:15.498+0000 7fb89488e700 1 --2- 172.21.15.32:0/3406510207 >> [v2:172.21.15.32:6800/3945405684,v1:172.21.15.32:6801/3945405684] conn(0x7fb87c03e420 0x7fb87c0408e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-03T18:37:20.072 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:17.098+0000 7fb89488e700 1 -- 172.21.15.32:0/3406510207 >> [v2:172.21.15.32:6800/3945405684,v1:172.21.15.32:6801/3945405684] conn(0x7fb87c03e420 msgr2=0x7fb87c0408e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6800/3945405684 2024-05-03T18:37:20.072 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:17.098+0000 7fb89488e700 1 --2- 172.21.15.32:0/3406510207 >> [v2:172.21.15.32:6800/3945405684,v1:172.21.15.32:6801/3945405684] conn(0x7fb87c03e420 0x7fb87c0408e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-05-03T18:37:20.072 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:18.492+0000 7fb8867fc700 1 -- 172.21.15.32:0/3406510207 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mgrmap(e 6) ==== 49989+0+0 (secure 0 0 0) 0x7fb880024770 con 0x7fb890107830 2024-05-03T18:37:20.072 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:18.492+0000 7fb8867fc700 1 -- 172.21.15.32:0/3406510207 >> [v2:172.21.15.32:6800/3945405684,v1:172.21.15.32:6801/3945405684] conn(0x7fb87c03e420 msgr2=0x7fb87c0408e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-03T18:37:20.072 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:18.492+0000 7fb8867fc700 1 --2- 172.21.15.32:0/3406510207 >> [v2:172.21.15.32:6800/3945405684,v1:172.21.15.32:6801/3945405684] conn(0x7fb87c03e420 0x7fb87c0408e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:20.072 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:19.493+0000 7fb8867fc700 1 -- 172.21.15.32:0/3406510207 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mgrmap(e 7) ==== 50116+0+0 (secure 0 0 0) 0x7fb880055c40 con 0x7fb890107830 2024-05-03T18:37:20.072 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:19.494+0000 7fb8867fc700 1 --2- 172.21.15.32:0/3406510207 >> [v2:172.21.15.32:6800/2019702098,v1:172.21.15.32:6801/2019702098] conn(0x7fb87c041d10 0x7fb87c044100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:20.072 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:19.494+0000 7fb8867fc700 1 -- 172.21.15.32:0/3406510207 --> [v2:172.21.15.32:6800/2019702098,v1:172.21.15.32:6801/2019702098] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fb87c040ff0 con 0x7fb87c041d10 2024-05-03T18:37:20.072 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:19.494+0000 7fb89488e700 1 --2- 172.21.15.32:0/3406510207 >> [v2:172.21.15.32:6800/2019702098,v1:172.21.15.32:6801/2019702098] conn(0x7fb87c041d10 0x7fb87c044100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:20.072 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:19.495+0000 7fb89488e700 1 --2- 172.21.15.32:0/3406510207 >> [v2:172.21.15.32:6800/2019702098,v1:172.21.15.32:6801/2019702098] conn(0x7fb87c041d10 0x7fb87c044100 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fb88c003a10 tx=0x7fb88c009290 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:37:20.072 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:19.495+0000 7fb8867fc700 1 -- 172.21.15.32:0/3406510207 <== mgr.14120 v2:172.21.15.32:6800/2019702098 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7fb87c040ff0 con 0x7fb87c041d10 2024-05-03T18:37:20.072 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:19.504+0000 7fb8972f3700 1 -- 172.21.15.32:0/3406510207 --> [v2:172.21.15.32:6800/2019702098,v1:172.21.15.32:6801/2019702098] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7fb890064b00 con 0x7fb87c041d10 2024-05-03T18:37:20.073 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:19.504+0000 7fb8867fc700 1 -- 172.21.15.32:0/3406510207 <== mgr.14120 v2:172.21.15.32:6800/2019702098 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7fb890064b00 con 0x7fb87c041d10 2024-05-03T18:37:20.073 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:19.504+0000 7fb8972f3700 1 -- 172.21.15.32:0/3406510207 >> [v2:172.21.15.32:6800/2019702098,v1:172.21.15.32:6801/2019702098] conn(0x7fb87c041d10 msgr2=0x7fb87c044100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:37:20.073 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:19.504+0000 7fb8972f3700 1 --2- 172.21.15.32:0/3406510207 >> [v2:172.21.15.32:6800/2019702098,v1:172.21.15.32:6801/2019702098] conn(0x7fb87c041d10 0x7fb87c044100 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fb88c003a10 tx=0x7fb88c009290 comp rx=0 tx=0).stop 2024-05-03T18:37:20.073 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:19.504+0000 7fb8972f3700 1 -- 172.21.15.32:0/3406510207 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb890107830 msgr2=0x7fb89019f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:37:20.073 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:19.504+0000 7fb8972f3700 1 --2- 172.21.15.32:0/3406510207 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb890107830 0x7fb89019f0d0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fb880003f40 tx=0x7fb880011890 comp rx=0 tx=0).stop 2024-05-03T18:37:20.073 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:19.504+0000 7fb8972f3700 1 -- 172.21.15.32:0/3406510207 shutdown_connections 2024-05-03T18:37:20.073 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:19.504+0000 7fb8972f3700 1 --2- 172.21.15.32:0/3406510207 >> [v2:172.21.15.32:6800/2019702098,v1:172.21.15.32:6801/2019702098] conn(0x7fb87c041d10 0x7fb87c044100 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:20.073 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:19.504+0000 7fb8972f3700 1 --2- 172.21.15.32:0/3406510207 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb890107830 0x7fb89019f0d0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:20.073 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:19.504+0000 7fb8972f3700 1 --2- 172.21.15.32:0/3406510207 >> [v2:172.21.15.32:6800/3945405684,v1:172.21.15.32:6801/3945405684] conn(0x7fb87c03e420 0x7fb87c0408e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:20.073 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:19.504+0000 7fb8972f3700 1 -- 172.21.15.32:0/3406510207 >> 172.21.15.32:0/3406510207 conn(0x7fb8901034d0 msgr2=0x7fb890195d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:37:20.073 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:19.504+0000 7fb8972f3700 1 -- 172.21.15.32:0/3406510207 shutdown_connections 2024-05-03T18:37:20.073 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:19.504+0000 7fb8972f3700 1 -- 172.21.15.32:0/3406510207 wait complete. 2024-05-03T18:37:20.073 INFO:teuthology.orchestra.run.smithi032.stdout:mgr epoch 5 is available 2024-05-03T18:37:20.073 INFO:teuthology.orchestra.run.smithi032.stdout:Setting orchestrator backend to cephadm... 2024-05-03T18:37:20.702 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:37:20 smithi032 ceph-mon[29763]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-05-03T18:37:20.702 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:37:20 smithi032 ceph-mon[29763]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-05-03T18:37:20.702 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:37:20 smithi032 ceph-mon[29763]: [03/May/2024:18:37:19] ENGINE Bus STARTING 2024-05-03T18:37:20.702 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:37:20 smithi032 ceph-mon[29763]: [03/May/2024:18:37:19] ENGINE Serving on http://172.21.15.32:8765 2024-05-03T18:37:20.702 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:37:20 smithi032 ceph-mon[29763]: [03/May/2024:18:37:19] ENGINE Serving on https://172.21.15.32:7150 2024-05-03T18:37:20.702 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:37:20 smithi032 ceph-mon[29763]: [03/May/2024:18:37:19] ENGINE Bus STARTED 2024-05-03T18:37:20.702 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:37:20 smithi032 ceph-mon[29763]: from='mgr.14120 172.21.15.32:0/2820864439' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:37:21.288 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:20.456+0000 7f1a827d2700 1 Processor -- start 2024-05-03T18:37:21.289 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:20.456+0000 7f1a827d2700 1 -- start start 2024-05-03T18:37:21.289 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:20.456+0000 7f1a827d2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1a7c109010 0x7f1a7c109430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:21.289 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:20.456+0000 7f1a827d2700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f1a7c109a00 con 0x7f1a7c109010 2024-05-03T18:37:21.289 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:20.456+0000 7f1a7bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1a7c109010 0x7f1a7c109430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:21.289 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:20.456+0000 7f1a7bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1a7c109010 0x7f1a7c109430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35550/0 (socket says 172.21.15.32:35550) 2024-05-03T18:37:21.289 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:20.456+0000 7f1a7bfff700 1 -- 172.21.15.32:0/117125534 learned_addr learned my addr 172.21.15.32:0/117125534 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:37:21.289 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:20.456+0000 7f1a7bfff700 1 -- 172.21.15.32:0/117125534 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a7c10a170 con 0x7f1a7c109010 2024-05-03T18:37:21.289 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:20.456+0000 7f1a7bfff700 1 --2- 172.21.15.32:0/117125534 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1a7c109010 0x7f1a7c109430 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1a6c0098a0 tx=0x7f1a6c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e64787fb2b4b5d4 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:37:21.289 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:20.457+0000 7f1a7affd700 1 -- 172.21.15.32:0/117125534 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1a6c004160 con 0x7f1a7c109010 2024-05-03T18:37:21.289 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:20.457+0000 7f1a7affd700 1 -- 172.21.15.32:0/117125534 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f1a6c004300 con 0x7f1a7c109010 2024-05-03T18:37:21.289 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:20.457+0000 7f1a7affd700 1 -- 172.21.15.32:0/117125534 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1a6c011af0 con 0x7f1a7c109010 2024-05-03T18:37:21.289 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:20.457+0000 7f1a827d2700 1 -- 172.21.15.32:0/117125534 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1a7c109010 msgr2=0x7f1a7c109430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:37:21.290 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:20.457+0000 7f1a827d2700 1 --2- 172.21.15.32:0/117125534 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1a7c109010 0x7f1a7c109430 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1a6c0098a0 tx=0x7f1a6c009bb0 comp rx=0 tx=0).stop 2024-05-03T18:37:21.290 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:20.457+0000 7f1a827d2700 1 -- 172.21.15.32:0/117125534 shutdown_connections 2024-05-03T18:37:21.290 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:20.457+0000 7f1a827d2700 1 --2- 172.21.15.32:0/117125534 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1a7c109010 0x7f1a7c109430 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:21.290 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:20.457+0000 7f1a827d2700 1 -- 172.21.15.32:0/117125534 >> 172.21.15.32:0/117125534 conn(0x7f1a7c104650 msgr2=0x7f1a7c106a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:37:21.290 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:20.457+0000 7f1a827d2700 1 -- 172.21.15.32:0/117125534 shutdown_connections 2024-05-03T18:37:21.290 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:20.457+0000 7f1a827d2700 1 -- 172.21.15.32:0/117125534 wait complete. 2024-05-03T18:37:21.290 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:20.458+0000 7f1a827d2700 1 Processor -- start 2024-05-03T18:37:21.290 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:20.458+0000 7f1a827d2700 1 -- start start 2024-05-03T18:37:21.290 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:20.458+0000 7f1a827d2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1a7c109010 0x7f1a7c19cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:21.290 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:20.458+0000 7f1a827d2700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f1a7c10d520 con 0x7f1a7c109010 2024-05-03T18:37:21.290 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:20.458+0000 7f1a7bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1a7c109010 0x7f1a7c19cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:21.290 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:20.458+0000 7f1a7bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1a7c109010 0x7f1a7c19cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35560/0 (socket says 172.21.15.32:35560) 2024-05-03T18:37:21.290 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:20.458+0000 7f1a7bfff700 1 -- 172.21.15.32:0/214168539 learned_addr learned my addr 172.21.15.32:0/214168539 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:37:21.290 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:20.459+0000 7f1a7bfff700 1 -- 172.21.15.32:0/214168539 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a7c19d480 con 0x7f1a7c109010 2024-05-03T18:37:21.290 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:20.459+0000 7f1a7bfff700 1 --2- 172.21.15.32:0/214168539 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1a7c109010 0x7f1a7c19cf40 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f1a6c0117b0 tx=0x7f1a6c011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:37:21.291 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:20.459+0000 7f1a797fa700 1 -- 172.21.15.32:0/214168539 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1a6c003af0 con 0x7f1a7c109010 2024-05-03T18:37:21.291 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:20.459+0000 7f1a797fa700 1 -- 172.21.15.32:0/214168539 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f1a6c020070 con 0x7f1a7c109010 2024-05-03T18:37:21.291 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:20.459+0000 7f1a827d2700 1 -- 172.21.15.32:0/214168539 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a7c19d710 con 0x7f1a7c109010 2024-05-03T18:37:21.291 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:20.459+0000 7f1a797fa700 1 -- 172.21.15.32:0/214168539 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1a6c019630 con 0x7f1a7c109010 2024-05-03T18:37:21.291 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:20.459+0000 7f1a827d2700 1 -- 172.21.15.32:0/214168539 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a7c1a0400 con 0x7f1a7c109010 2024-05-03T18:37:21.291 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:20.460+0000 7f1a827d2700 1 -- 172.21.15.32:0/214168539 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a7c064b00 con 0x7f1a7c109010 2024-05-03T18:37:21.291 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:20.460+0000 7f1a797fa700 1 -- 172.21.15.32:0/214168539 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 7) ==== 50116+0+0 (secure 0 0 0) 0x7f1a6c01a9c0 con 0x7f1a7c109010 2024-05-03T18:37:21.291 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:20.460+0000 7f1a797fa700 1 --2- 172.21.15.32:0/214168539 >> [v2:172.21.15.32:6800/2019702098,v1:172.21.15.32:6801/2019702098] conn(0x7f1a6403e350 0x7f1a64040810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:21.291 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:20.460+0000 7f1a797fa700 1 -- 172.21.15.32:0/214168539 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f1a6c053920 con 0x7f1a7c109010 2024-05-03T18:37:21.291 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:20.460+0000 7f1a7b7fe700 1 --2- 172.21.15.32:0/214168539 >> [v2:172.21.15.32:6800/2019702098,v1:172.21.15.32:6801/2019702098] conn(0x7f1a6403e350 0x7f1a64040810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:21.291 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:20.461+0000 7f1a7b7fe700 1 --2- 172.21.15.32:0/214168539 >> [v2:172.21.15.32:6800/2019702098,v1:172.21.15.32:6801/2019702098] conn(0x7f1a6403e350 0x7f1a64040810 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f1a70006fd0 tx=0x7f1a70006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:37:21.291 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:20.466+0000 7f1a797fa700 1 -- 172.21.15.32:0/214168539 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f1a6c0172b0 con 0x7f1a7c109010 2024-05-03T18:37:21.291 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:20.546+0000 7f1a797fa700 1 -- 172.21.15.32:0/214168539 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f1a6c01a430 con 0x7f1a7c109010 2024-05-03T18:37:21.291 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:20.700+0000 7f1a827d2700 1 -- 172.21.15.32:0/214168539 --> [v2:172.21.15.32:6800/2019702098,v1:172.21.15.32:6801/2019702098] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f1a7c1058f0 con 0x7f1a6403e350 2024-05-03T18:37:21.291 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:20.705+0000 7f1a797fa700 1 -- 172.21.15.32:0/214168539 <== mgr.14120 v2:172.21.15.32:6800/2019702098 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f1a7c1058f0 con 0x7f1a6403e350 2024-05-03T18:37:21.292 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:20.708+0000 7f1a827d2700 1 -- 172.21.15.32:0/214168539 >> [v2:172.21.15.32:6800/2019702098,v1:172.21.15.32:6801/2019702098] conn(0x7f1a6403e350 msgr2=0x7f1a64040810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:37:21.292 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:20.708+0000 7f1a827d2700 1 --2- 172.21.15.32:0/214168539 >> [v2:172.21.15.32:6800/2019702098,v1:172.21.15.32:6801/2019702098] conn(0x7f1a6403e350 0x7f1a64040810 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f1a70006fd0 tx=0x7f1a70006e40 comp rx=0 tx=0).stop 2024-05-03T18:37:21.292 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:20.708+0000 7f1a827d2700 1 -- 172.21.15.32:0/214168539 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1a7c109010 msgr2=0x7f1a7c19cf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:37:21.292 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:20.708+0000 7f1a827d2700 1 --2- 172.21.15.32:0/214168539 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1a7c109010 0x7f1a7c19cf40 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f1a6c0117b0 tx=0x7f1a6c011b70 comp rx=0 tx=0).stop 2024-05-03T18:37:21.292 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:20.708+0000 7f1a827d2700 1 -- 172.21.15.32:0/214168539 shutdown_connections 2024-05-03T18:37:21.292 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:20.708+0000 7f1a827d2700 1 --2- 172.21.15.32:0/214168539 >> [v2:172.21.15.32:6800/2019702098,v1:172.21.15.32:6801/2019702098] conn(0x7f1a6403e350 0x7f1a64040810 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:21.292 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:20.708+0000 7f1a827d2700 1 --2- 172.21.15.32:0/214168539 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1a7c109010 0x7f1a7c19cf40 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:21.292 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:20.708+0000 7f1a827d2700 1 -- 172.21.15.32:0/214168539 >> 172.21.15.32:0/214168539 conn(0x7f1a7c104650 msgr2=0x7f1a7c1051e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:37:21.292 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:20.708+0000 7f1a827d2700 1 -- 172.21.15.32:0/214168539 shutdown_connections 2024-05-03T18:37:21.292 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:20.708+0000 7f1a827d2700 1 -- 172.21.15.32:0/214168539 wait complete. 2024-05-03T18:37:21.817 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:37:21 smithi032 ceph-mon[29763]: mgrmap e8: x(active, since 2s) 2024-05-03T18:37:21.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:37:21 smithi032 ceph-mon[29763]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:37:21.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:37:21 smithi032 ceph-mon[29763]: from='mgr.14120 172.21.15.32:0/2820864439' entity='mgr.x' 2024-05-03T18:37:21.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:37:21 smithi032 ceph-mon[29763]: from='mgr.14120 172.21.15.32:0/2820864439' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:37:22.504 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout value unchanged 2024-05-03T18:37:22.504 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:21.693+0000 7ff27eb18700 1 Processor -- start 2024-05-03T18:37:22.504 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:21.693+0000 7ff27eb18700 1 -- start start 2024-05-03T18:37:22.504 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:21.693+0000 7ff27eb18700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff278109010 0x7ff278109430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:22.504 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:21.693+0000 7ff27eb18700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff278109a00 con 0x7ff278109010 2024-05-03T18:37:22.504 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:21.694+0000 7ff27c8b4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff278109010 0x7ff278109430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:22.504 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:21.694+0000 7ff27c8b4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff278109010 0x7ff278109430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35566/0 (socket says 172.21.15.32:35566) 2024-05-03T18:37:22.505 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:21.694+0000 7ff27c8b4700 1 -- 172.21.15.32:0/1844698733 learned_addr learned my addr 172.21.15.32:0/1844698733 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:37:22.505 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:21.694+0000 7ff27c8b4700 1 -- 172.21.15.32:0/1844698733 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff27810a170 con 0x7ff278109010 2024-05-03T18:37:22.505 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:21.694+0000 7ff27c8b4700 1 --2- 172.21.15.32:0/1844698733 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff278109010 0x7ff278109430 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7ff270009e80 tx=0x7ff27000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2931f9967b2ad68f server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:37:22.505 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:21.694+0000 7ff2777fe700 1 -- 172.21.15.32:0/1844698733 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff270017070 con 0x7ff278109010 2024-05-03T18:37:22.505 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:21.694+0000 7ff2777fe700 1 -- 172.21.15.32:0/1844698733 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7ff27000b8c0 con 0x7ff278109010 2024-05-03T18:37:22.505 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:21.694+0000 7ff2777fe700 1 -- 172.21.15.32:0/1844698733 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff270004490 con 0x7ff278109010 2024-05-03T18:37:22.505 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:21.695+0000 7ff27eb18700 1 -- 172.21.15.32:0/1844698733 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff278109010 msgr2=0x7ff278109430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:37:22.505 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:21.695+0000 7ff27eb18700 1 --2- 172.21.15.32:0/1844698733 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff278109010 0x7ff278109430 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7ff270009e80 tx=0x7ff27000b330 comp rx=0 tx=0).stop 2024-05-03T18:37:22.505 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:21.695+0000 7ff27eb18700 1 -- 172.21.15.32:0/1844698733 shutdown_connections 2024-05-03T18:37:22.505 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:21.695+0000 7ff27eb18700 1 --2- 172.21.15.32:0/1844698733 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff278109010 0x7ff278109430 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:22.505 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:21.695+0000 7ff27eb18700 1 -- 172.21.15.32:0/1844698733 >> 172.21.15.32:0/1844698733 conn(0x7ff278104650 msgr2=0x7ff278106a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:37:22.505 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:21.695+0000 7ff27eb18700 1 -- 172.21.15.32:0/1844698733 shutdown_connections 2024-05-03T18:37:22.505 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:21.695+0000 7ff27eb18700 1 -- 172.21.15.32:0/1844698733 wait complete. 2024-05-03T18:37:22.505 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:21.696+0000 7ff27eb18700 1 Processor -- start 2024-05-03T18:37:22.506 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:21.696+0000 7ff27eb18700 1 -- start start 2024-05-03T18:37:22.506 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:21.696+0000 7ff27eb18700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff278109010 0x7ff27807eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:22.506 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:21.696+0000 7ff27eb18700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff27810d520 con 0x7ff278109010 2024-05-03T18:37:22.506 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:21.696+0000 7ff27c8b4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff278109010 0x7ff27807eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:22.506 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:21.696+0000 7ff27c8b4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff278109010 0x7ff27807eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35570/0 (socket says 172.21.15.32:35570) 2024-05-03T18:37:22.506 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:21.696+0000 7ff27c8b4700 1 -- 172.21.15.32:0/4177245146 learned_addr learned my addr 172.21.15.32:0/4177245146 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:37:22.506 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:21.696+0000 7ff27c8b4700 1 -- 172.21.15.32:0/4177245146 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff27807f0c0 con 0x7ff278109010 2024-05-03T18:37:22.506 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:21.696+0000 7ff27c8b4700 1 --2- 172.21.15.32:0/4177245146 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff278109010 0x7ff27807eb80 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff27000b7b0 tx=0x7ff27000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:37:22.506 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:21.696+0000 7ff275ffb700 1 -- 172.21.15.32:0/4177245146 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff270020020 con 0x7ff278109010 2024-05-03T18:37:22.506 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:21.697+0000 7ff275ffb700 1 -- 172.21.15.32:0/4177245146 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7ff270012430 con 0x7ff278109010 2024-05-03T18:37:22.506 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:21.697+0000 7ff275ffb700 1 -- 172.21.15.32:0/4177245146 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff270017040 con 0x7ff278109010 2024-05-03T18:37:22.506 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:21.697+0000 7ff27eb18700 1 -- 172.21.15.32:0/4177245146 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff27807d2b0 con 0x7ff278109010 2024-05-03T18:37:22.506 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:21.697+0000 7ff27eb18700 1 -- 172.21.15.32:0/4177245146 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff27807d7f0 con 0x7ff278109010 2024-05-03T18:37:22.506 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:21.698+0000 7ff275ffb700 1 -- 172.21.15.32:0/4177245146 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7ff270023020 con 0x7ff278109010 2024-05-03T18:37:22.507 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:21.698+0000 7ff27eb18700 1 -- 172.21.15.32:0/4177245146 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff278064b00 con 0x7ff278109010 2024-05-03T18:37:22.507 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:21.698+0000 7ff275ffb700 1 --2- 172.21.15.32:0/4177245146 >> [v2:172.21.15.32:6800/2019702098,v1:172.21.15.32:6801/2019702098] conn(0x7ff26403e420 0x7ff2640408e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:22.507 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:21.698+0000 7ff275ffb700 1 -- 172.21.15.32:0/4177245146 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7ff27000f050 con 0x7ff278109010 2024-05-03T18:37:22.507 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:21.698+0000 7ff277fff700 1 --2- 172.21.15.32:0/4177245146 >> [v2:172.21.15.32:6800/2019702098,v1:172.21.15.32:6801/2019702098] conn(0x7ff26403e420 0x7ff2640408e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:22.507 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:21.698+0000 7ff277fff700 1 --2- 172.21.15.32:0/4177245146 >> [v2:172.21.15.32:6800/2019702098,v1:172.21.15.32:6801/2019702098] conn(0x7ff26403e420 0x7ff2640408e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff268006fd0 tx=0x7ff268006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:37:22.507 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:21.703+0000 7ff275ffb700 1 -- 172.21.15.32:0/4177245146 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7ff27002a020 con 0x7ff278109010 2024-05-03T18:37:22.507 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:21.919+0000 7ff27eb18700 1 -- 172.21.15.32:0/4177245146 --> [v2:172.21.15.32:6800/2019702098,v1:172.21.15.32:6801/2019702098] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7ff278105850 con 0x7ff26403e420 2024-05-03T18:37:22.507 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:21.919+0000 7ff275ffb700 1 -- 172.21.15.32:0/4177245146 <== mgr.14120 v2:172.21.15.32:6800/2019702098 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7ff278105850 con 0x7ff26403e420 2024-05-03T18:37:22.507 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:21.922+0000 7ff27eb18700 1 -- 172.21.15.32:0/4177245146 >> [v2:172.21.15.32:6800/2019702098,v1:172.21.15.32:6801/2019702098] conn(0x7ff26403e420 msgr2=0x7ff2640408e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:37:22.507 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:21.922+0000 7ff27eb18700 1 --2- 172.21.15.32:0/4177245146 >> [v2:172.21.15.32:6800/2019702098,v1:172.21.15.32:6801/2019702098] conn(0x7ff26403e420 0x7ff2640408e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff268006fd0 tx=0x7ff268006e40 comp rx=0 tx=0).stop 2024-05-03T18:37:22.507 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:21.922+0000 7ff27eb18700 1 -- 172.21.15.32:0/4177245146 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff278109010 msgr2=0x7ff27807eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:37:22.507 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:21.922+0000 7ff27eb18700 1 --2- 172.21.15.32:0/4177245146 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff278109010 0x7ff27807eb80 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff27000b7b0 tx=0x7ff27000be20 comp rx=0 tx=0).stop 2024-05-03T18:37:22.507 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:21.922+0000 7ff27eb18700 1 -- 172.21.15.32:0/4177245146 shutdown_connections 2024-05-03T18:37:22.508 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:21.922+0000 7ff27eb18700 1 --2- 172.21.15.32:0/4177245146 >> [v2:172.21.15.32:6800/2019702098,v1:172.21.15.32:6801/2019702098] conn(0x7ff26403e420 0x7ff2640408e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:22.508 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:21.922+0000 7ff27eb18700 1 --2- 172.21.15.32:0/4177245146 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff278109010 0x7ff27807eb80 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:22.508 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:21.922+0000 7ff27eb18700 1 -- 172.21.15.32:0/4177245146 >> 172.21.15.32:0/4177245146 conn(0x7ff278104650 msgr2=0x7ff278105140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:37:22.508 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:21.922+0000 7ff27eb18700 1 -- 172.21.15.32:0/4177245146 shutdown_connections 2024-05-03T18:37:22.508 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:21.922+0000 7ff27eb18700 1 -- 172.21.15.32:0/4177245146 wait complete. 2024-05-03T18:37:22.508 INFO:teuthology.orchestra.run.smithi032.stdout:Generating ssh key... 2024-05-03T18:37:22.790 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:37:22 smithi032 ceph-mon[29763]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:37:23.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:37:23 smithi032 ceph-mon[29763]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:37:23.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:37:23 smithi032 ceph-mon[29763]: Generating ssh key... 2024-05-03T18:37:23.818 INFO:journalctl@ceph.mgr.x.smithi032.stdout:May 03 18:37:23 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-x[29995]: Generating public/private rsa key pair. 2024-05-03T18:37:23.818 INFO:journalctl@ceph.mgr.x.smithi032.stdout:May 03 18:37:23 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-x[29995]: Your identification has been saved in /tmp/tmp_hk_guo5/key. 2024-05-03T18:37:23.818 INFO:journalctl@ceph.mgr.x.smithi032.stdout:May 03 18:37:23 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-x[29995]: Your public key has been saved in /tmp/tmp_hk_guo5/key.pub. 2024-05-03T18:37:23.818 INFO:journalctl@ceph.mgr.x.smithi032.stdout:May 03 18:37:23 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-x[29995]: The key fingerprint is: 2024-05-03T18:37:23.818 INFO:journalctl@ceph.mgr.x.smithi032.stdout:May 03 18:37:23 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-x[29995]: SHA256:hBGJ5SBJ9SfimITfC7L9UF8pigFjZTVr7hP0/h5bSYI ceph-f500455e-097b-11ef-bc97-c7b262605968 2024-05-03T18:37:23.819 INFO:journalctl@ceph.mgr.x.smithi032.stdout:May 03 18:37:23 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-x[29995]: The key's randomart image is: 2024-05-03T18:37:23.819 INFO:journalctl@ceph.mgr.x.smithi032.stdout:May 03 18:37:23 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-x[29995]: +---[RSA 3072]----+ 2024-05-03T18:37:23.819 INFO:journalctl@ceph.mgr.x.smithi032.stdout:May 03 18:37:23 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-x[29995]: | .o=+=+o | 2024-05-03T18:37:23.819 INFO:journalctl@ceph.mgr.x.smithi032.stdout:May 03 18:37:23 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-x[29995]: | .+..++o | 2024-05-03T18:37:23.819 INFO:journalctl@ceph.mgr.x.smithi032.stdout:May 03 18:37:23 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-x[29995]: |+.. .+= o | 2024-05-03T18:37:23.819 INFO:journalctl@ceph.mgr.x.smithi032.stdout:May 03 18:37:23 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-x[29995]: |.= =+..+.. | 2024-05-03T18:37:23.819 INFO:journalctl@ceph.mgr.x.smithi032.stdout:May 03 18:37:23 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-x[29995]: |. * +o.ES. . | 2024-05-03T18:37:23.819 INFO:journalctl@ceph.mgr.x.smithi032.stdout:May 03 18:37:23 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-x[29995]: | + =.+oo o . | 2024-05-03T18:37:23.819 INFO:journalctl@ceph.mgr.x.smithi032.stdout:May 03 18:37:23 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-x[29995]: |. + oo.. . o | 2024-05-03T18:37:23.819 INFO:journalctl@ceph.mgr.x.smithi032.stdout:May 03 18:37:23 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-x[29995]: | o . . + | 2024-05-03T18:37:23.819 INFO:journalctl@ceph.mgr.x.smithi032.stdout:May 03 18:37:23 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-x[29995]: | . .+ | 2024-05-03T18:37:23.819 INFO:journalctl@ceph.mgr.x.smithi032.stdout:May 03 18:37:23 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-x[29995]: +----[SHA256]-----+ 2024-05-03T18:37:24.317 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:22.901+0000 7fb2c3c38700 1 Processor -- start 2024-05-03T18:37:24.318 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:22.902+0000 7fb2c3c38700 1 -- start start 2024-05-03T18:37:24.318 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:22.902+0000 7fb2c3c38700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb2bc1072b0 0x7fb2bc1076d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:24.318 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:22.902+0000 7fb2c3c38700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fb2bc107ca0 con 0x7fb2bc1072b0 2024-05-03T18:37:24.318 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:22.902+0000 7fb2c19d4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb2bc1072b0 0x7fb2bc1076d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:24.318 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:22.902+0000 7fb2c19d4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb2bc1072b0 0x7fb2bc1076d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35576/0 (socket says 172.21.15.32:35576) 2024-05-03T18:37:24.318 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:22.902+0000 7fb2c19d4700 1 -- 172.21.15.32:0/671550158 learned_addr learned my addr 172.21.15.32:0/671550158 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:37:24.318 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:22.902+0000 7fb2c19d4700 1 -- 172.21.15.32:0/671550158 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2bc10a240 con 0x7fb2bc1072b0 2024-05-03T18:37:24.318 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:22.903+0000 7fb2c19d4700 1 --2- 172.21.15.32:0/671550158 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb2bc1072b0 0x7fb2bc1076d0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fb2ac0098a0 tx=0x7fb2ac009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b276a0c6ca9312c6 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:37:24.318 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:22.903+0000 7fb2c09d2700 1 -- 172.21.15.32:0/671550158 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb2ac004160 con 0x7fb2bc1072b0 2024-05-03T18:37:24.318 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:22.903+0000 7fb2c09d2700 1 -- 172.21.15.32:0/671550158 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fb2ac004300 con 0x7fb2bc1072b0 2024-05-03T18:37:24.318 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:22.903+0000 7fb2c09d2700 1 -- 172.21.15.32:0/671550158 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb2ac011af0 con 0x7fb2bc1072b0 2024-05-03T18:37:24.318 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:22.903+0000 7fb2c3c38700 1 -- 172.21.15.32:0/671550158 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb2bc1072b0 msgr2=0x7fb2bc1076d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:37:24.319 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:22.903+0000 7fb2c3c38700 1 --2- 172.21.15.32:0/671550158 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb2bc1072b0 0x7fb2bc1076d0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fb2ac0098a0 tx=0x7fb2ac009bb0 comp rx=0 tx=0).stop 2024-05-03T18:37:24.319 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:22.903+0000 7fb2c3c38700 1 -- 172.21.15.32:0/671550158 shutdown_connections 2024-05-03T18:37:24.319 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:22.903+0000 7fb2c3c38700 1 --2- 172.21.15.32:0/671550158 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb2bc1072b0 0x7fb2bc1076d0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:24.319 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:22.903+0000 7fb2c3c38700 1 -- 172.21.15.32:0/671550158 >> 172.21.15.32:0/671550158 conn(0x7fb2bc102c60 msgr2=0x7fb2bc105080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:37:24.319 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:22.904+0000 7fb2c3c38700 1 -- 172.21.15.32:0/671550158 shutdown_connections 2024-05-03T18:37:24.319 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:22.904+0000 7fb2c3c38700 1 -- 172.21.15.32:0/671550158 wait complete. 2024-05-03T18:37:24.319 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:22.904+0000 7fb2c3c38700 1 Processor -- start 2024-05-03T18:37:24.319 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:22.904+0000 7fb2c3c38700 1 -- start start 2024-05-03T18:37:24.319 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:22.904+0000 7fb2c3c38700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb2bc1072b0 0x7fb2bc19cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:24.319 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:22.904+0000 7fb2c3c38700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fb2bc10d5f0 con 0x7fb2bc1072b0 2024-05-03T18:37:24.319 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:22.905+0000 7fb2c19d4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb2bc1072b0 0x7fb2bc19cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:24.319 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:22.905+0000 7fb2c19d4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb2bc1072b0 0x7fb2bc19cf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35592/0 (socket says 172.21.15.32:35592) 2024-05-03T18:37:24.319 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:22.905+0000 7fb2c19d4700 1 -- 172.21.15.32:0/3437962721 learned_addr learned my addr 172.21.15.32:0/3437962721 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:37:24.319 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:22.905+0000 7fb2c19d4700 1 -- 172.21.15.32:0/3437962721 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2bc19d4c0 con 0x7fb2bc1072b0 2024-05-03T18:37:24.319 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:22.905+0000 7fb2c19d4700 1 --2- 172.21.15.32:0/3437962721 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb2bc1072b0 0x7fb2bc19cf80 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb2ac000c00 tx=0x7fb2ac011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:37:24.320 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:22.905+0000 7fb2b2ffd700 1 -- 172.21.15.32:0/3437962721 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb2ac003b20 con 0x7fb2bc1072b0 2024-05-03T18:37:24.320 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:22.905+0000 7fb2b2ffd700 1 -- 172.21.15.32:0/3437962721 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fb2ac01a8c0 con 0x7fb2bc1072b0 2024-05-03T18:37:24.320 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:22.905+0000 7fb2b2ffd700 1 -- 172.21.15.32:0/3437962721 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb2ac024940 con 0x7fb2bc1072b0 2024-05-03T18:37:24.320 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:22.905+0000 7fb2c3c38700 1 -- 172.21.15.32:0/3437962721 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2bc19d750 con 0x7fb2bc1072b0 2024-05-03T18:37:24.320 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:22.905+0000 7fb2c3c38700 1 -- 172.21.15.32:0/3437962721 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2bc1a03f0 con 0x7fb2bc1072b0 2024-05-03T18:37:24.320 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:22.906+0000 7fb2b2ffd700 1 -- 172.21.15.32:0/3437962721 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7fb2ac022020 con 0x7fb2bc1072b0 2024-05-03T18:37:24.320 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:22.906+0000 7fb2c3c38700 1 -- 172.21.15.32:0/3437962721 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2bc064b00 con 0x7fb2bc1072b0 2024-05-03T18:37:24.320 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:22.906+0000 7fb2b2ffd700 1 --2- 172.21.15.32:0/3437962721 >> [v2:172.21.15.32:6800/2019702098,v1:172.21.15.32:6801/2019702098] conn(0x7fb2a803e4c0 0x7fb2a8040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:24.320 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:22.907+0000 7fb2b2ffd700 1 -- 172.21.15.32:0/3437962721 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fb2ac053750 con 0x7fb2bc1072b0 2024-05-03T18:37:24.320 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:22.907+0000 7fb2c11d3700 1 --2- 172.21.15.32:0/3437962721 >> [v2:172.21.15.32:6800/2019702098,v1:172.21.15.32:6801/2019702098] conn(0x7fb2a803e4c0 0x7fb2a8040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:24.320 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:22.907+0000 7fb2c11d3700 1 --2- 172.21.15.32:0/3437962721 >> [v2:172.21.15.32:6800/2019702098,v1:172.21.15.32:6801/2019702098] conn(0x7fb2a803e4c0 0x7fb2a8040980 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb2b8006fd0 tx=0x7fb2b8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:37:24.320 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:22.912+0000 7fb2b2ffd700 1 -- 172.21.15.32:0/3437962721 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fb2ac01aa70 con 0x7fb2bc1072b0 2024-05-03T18:37:24.320 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:23.135+0000 7fb2c3c38700 1 -- 172.21.15.32:0/3437962721 --> [v2:172.21.15.32:6800/2019702098,v1:172.21.15.32:6801/2019702098] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7fb2bc103eb0 con 0x7fb2a803e4c0 2024-05-03T18:37:24.320 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:23.768+0000 7fb2b2ffd700 1 -- 172.21.15.32:0/3437962721 <== mgr.14120 v2:172.21.15.32:6800/2019702098 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fb2bc103eb0 con 0x7fb2a803e4c0 2024-05-03T18:37:24.320 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:23.771+0000 7fb2c3c38700 1 -- 172.21.15.32:0/3437962721 >> [v2:172.21.15.32:6800/2019702098,v1:172.21.15.32:6801/2019702098] conn(0x7fb2a803e4c0 msgr2=0x7fb2a8040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:37:24.321 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:23.771+0000 7fb2c3c38700 1 --2- 172.21.15.32:0/3437962721 >> [v2:172.21.15.32:6800/2019702098,v1:172.21.15.32:6801/2019702098] conn(0x7fb2a803e4c0 0x7fb2a8040980 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb2b8006fd0 tx=0x7fb2b8006e40 comp rx=0 tx=0).stop 2024-05-03T18:37:24.321 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:23.771+0000 7fb2c3c38700 1 -- 172.21.15.32:0/3437962721 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb2bc1072b0 msgr2=0x7fb2bc19cf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:37:24.321 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:23.771+0000 7fb2c3c38700 1 --2- 172.21.15.32:0/3437962721 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb2bc1072b0 0x7fb2bc19cf80 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb2ac000c00 tx=0x7fb2ac011b70 comp rx=0 tx=0).stop 2024-05-03T18:37:24.321 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:23.771+0000 7fb2c3c38700 1 -- 172.21.15.32:0/3437962721 shutdown_connections 2024-05-03T18:37:24.321 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:23.771+0000 7fb2c3c38700 1 --2- 172.21.15.32:0/3437962721 >> [v2:172.21.15.32:6800/2019702098,v1:172.21.15.32:6801/2019702098] conn(0x7fb2a803e4c0 0x7fb2a8040980 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:24.321 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:23.771+0000 7fb2c3c38700 1 --2- 172.21.15.32:0/3437962721 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb2bc1072b0 0x7fb2bc19cf80 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:24.321 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:23.771+0000 7fb2c3c38700 1 -- 172.21.15.32:0/3437962721 >> 172.21.15.32:0/3437962721 conn(0x7fb2bc102c60 msgr2=0x7fb2bc1037a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:37:24.321 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:23.771+0000 7fb2c3c38700 1 -- 172.21.15.32:0/3437962721 shutdown_connections 2024-05-03T18:37:24.321 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:23.771+0000 7fb2c3c38700 1 -- 172.21.15.32:0/3437962721 wait complete. 2024-05-03T18:37:25.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:37:24 smithi032 ceph-mon[29763]: from='mgr.14120 172.21.15.32:0/2820864439' entity='mgr.x' 2024-05-03T18:37:25.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:37:24 smithi032 ceph-mon[29763]: from='mgr.14120 172.21.15.32:0/2820864439' entity='mgr.x' 2024-05-03T18:37:25.491 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC+oFN0k/X5PxHqlAC26+EPkxhb41cOmEL7M6meXPdUq98nXACJJSdiLyb4jF1uj4IiucK6o6m+0+HF+RctoQfRue4dqzM+Aqc4aoomOevP9qoeBukZkanqw1ja/shNVJoOnQUP9P0XYCI0Y4GQ2VGuvmVN8KGhocbiNaweG3kfhSj+YABVB47Mafot3iv+lMt0pd1hgZZfgb4DbC8T5tvu1rVurzBWrr0NU5de4C0ja2a7J+d7ds+9PRa4BmCZ++4ZwBNXn6G0kgl6jM2cvCuURY8ulL1tmSdJP1WRXRV0HIIQbo6CL0R2V+3Idfu5h6VYvyVN/i/ZaXu0IkkrSKtUO5cr6UC9yIn2096yV/8w5KoU7J0Q29fe9J8le4VBHHbP/w55/1sdrPht8RpyA4ER+zQyX1BknR578EE5knbuEd/s790NA6pm/edy2gTNz2b56uTbOTVQlt1ViEZL96c31MAHYX71AUQVQrd2pw8DAxpZYaipivTvFLMTOTKgORE= ceph-f500455e-097b-11ef-bc97-c7b262605968 2024-05-03T18:37:25.491 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:24.734+0000 7f04ee4a6700 1 Processor -- start 2024-05-03T18:37:25.492 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:24.734+0000 7f04ee4a6700 1 -- start start 2024-05-03T18:37:25.492 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:24.734+0000 7f04ee4a6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04e807ee30 0x7f04e807d330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:25.492 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:24.734+0000 7f04ee4a6700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f04e807d870 con 0x7f04e807ee30 2024-05-03T18:37:25.492 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:24.735+0000 7f04e7fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04e807ee30 0x7f04e807d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:25.492 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:24.735+0000 7f04e7fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04e807ee30 0x7f04e807d330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53100/0 (socket says 172.21.15.32:53100) 2024-05-03T18:37:25.492 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:24.735+0000 7f04e7fff700 1 -- 172.21.15.32:0/3625177741 learned_addr learned my addr 172.21.15.32:0/3625177741 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:37:25.492 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:24.735+0000 7f04e7fff700 1 -- 172.21.15.32:0/3625177741 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04e807d9f0 con 0x7f04e807ee30 2024-05-03T18:37:25.492 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:24.735+0000 7f04e7fff700 1 --2- 172.21.15.32:0/3625177741 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04e807ee30 0x7f04e807d330 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f04dc0098a0 tx=0x7f04dc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22f40f057e3ccf3e server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:37:25.492 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:24.735+0000 7f04e6ffd700 1 -- 172.21.15.32:0/3625177741 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f04dc004160 con 0x7f04e807ee30 2024-05-03T18:37:25.492 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:24.735+0000 7f04e6ffd700 1 -- 172.21.15.32:0/3625177741 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f04dc004300 con 0x7f04e807ee30 2024-05-03T18:37:25.493 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:24.735+0000 7f04e6ffd700 1 -- 172.21.15.32:0/3625177741 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f04dc011af0 con 0x7f04e807ee30 2024-05-03T18:37:25.493 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:24.736+0000 7f04ee4a6700 1 -- 172.21.15.32:0/3625177741 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04e807ee30 msgr2=0x7f04e807d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:37:25.493 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:24.736+0000 7f04ee4a6700 1 --2- 172.21.15.32:0/3625177741 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04e807ee30 0x7f04e807d330 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f04dc0098a0 tx=0x7f04dc009bb0 comp rx=0 tx=0).stop 2024-05-03T18:37:25.493 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:24.736+0000 7f04ee4a6700 1 -- 172.21.15.32:0/3625177741 shutdown_connections 2024-05-03T18:37:25.493 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:24.736+0000 7f04ee4a6700 1 --2- 172.21.15.32:0/3625177741 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04e807ee30 0x7f04e807d330 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:25.493 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:24.736+0000 7f04ee4a6700 1 -- 172.21.15.32:0/3625177741 >> 172.21.15.32:0/3625177741 conn(0x7f04e8104650 msgr2=0x7f04e8106a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:37:25.494 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:24.736+0000 7f04ee4a6700 1 -- 172.21.15.32:0/3625177741 shutdown_connections 2024-05-03T18:37:25.494 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:24.736+0000 7f04ee4a6700 1 -- 172.21.15.32:0/3625177741 wait complete. 2024-05-03T18:37:25.494 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:24.736+0000 7f04ee4a6700 1 Processor -- start 2024-05-03T18:37:25.494 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:24.737+0000 7f04ee4a6700 1 -- start start 2024-05-03T18:37:25.494 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:24.737+0000 7f04ee4a6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04e807ee30 0x7f04e819f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:25.494 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:24.737+0000 7f04ee4a6700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f04e810f540 con 0x7f04e807ee30 2024-05-03T18:37:25.495 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:24.737+0000 7f04e7fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04e807ee30 0x7f04e819f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:25.495 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:24.737+0000 7f04e7fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04e807ee30 0x7f04e819f100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53104/0 (socket says 172.21.15.32:53104) 2024-05-03T18:37:25.495 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:24.737+0000 7f04e7fff700 1 -- 172.21.15.32:0/3740738237 learned_addr learned my addr 172.21.15.32:0/3740738237 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:37:25.495 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:24.737+0000 7f04e7fff700 1 -- 172.21.15.32:0/3740738237 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04e819f640 con 0x7f04e807ee30 2024-05-03T18:37:25.495 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:24.737+0000 7f04e7fff700 1 --2- 172.21.15.32:0/3740738237 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04e807ee30 0x7f04e819f100 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f04dc011ac0 tx=0x7f04dc011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:37:25.495 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:24.737+0000 7f04e57fa700 1 -- 172.21.15.32:0/3740738237 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f04dc003970 con 0x7f04e807ee30 2024-05-03T18:37:25.495 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:24.738+0000 7f04ee4a6700 1 -- 172.21.15.32:0/3740738237 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04e819f8d0 con 0x7f04e807ee30 2024-05-03T18:37:25.495 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:24.738+0000 7f04e57fa700 1 -- 172.21.15.32:0/3740738237 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f04dc003b10 con 0x7f04e807ee30 2024-05-03T18:37:25.496 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:24.738+0000 7f04e57fa700 1 -- 172.21.15.32:0/3740738237 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f04dc024610 con 0x7f04e807ee30 2024-05-03T18:37:25.496 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:24.738+0000 7f04ee4a6700 1 -- 172.21.15.32:0/3740738237 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04e81a25c0 con 0x7f04e807ee30 2024-05-03T18:37:25.496 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:24.738+0000 7f04e57fa700 1 -- 172.21.15.32:0/3740738237 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f04dc024890 con 0x7f04e807ee30 2024-05-03T18:37:25.496 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:24.739+0000 7f04e57fa700 1 --2- 172.21.15.32:0/3740738237 >> [v2:172.21.15.32:6800/2019702098,v1:172.21.15.32:6801/2019702098] conn(0x7f04d003e470 0x7f04d0040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:25.496 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:24.739+0000 7f04e57fa700 1 -- 172.21.15.32:0/3740738237 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f04dc053b70 con 0x7f04e807ee30 2024-05-03T18:37:25.496 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:24.739+0000 7f04e77fe700 1 --2- 172.21.15.32:0/3740738237 >> [v2:172.21.15.32:6800/2019702098,v1:172.21.15.32:6801/2019702098] conn(0x7f04d003e470 0x7f04d0040930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:25.496 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:24.739+0000 7f04ee4a6700 1 -- 172.21.15.32:0/3740738237 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04e8064b00 con 0x7f04e807ee30 2024-05-03T18:37:25.496 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:24.739+0000 7f04e77fe700 1 --2- 172.21.15.32:0/3740738237 >> [v2:172.21.15.32:6800/2019702098,v1:172.21.15.32:6801/2019702098] conn(0x7f04d003e470 0x7f04d0040930 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f04d8006fd0 tx=0x7f04d8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:37:25.497 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:24.744+0000 7f04e57fa700 1 -- 172.21.15.32:0/3740738237 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f04dc02c350 con 0x7f04e807ee30 2024-05-03T18:37:25.497 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:24.959+0000 7f04ee4a6700 1 -- 172.21.15.32:0/3740738237 --> [v2:172.21.15.32:6800/2019702098,v1:172.21.15.32:6801/2019702098] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f04e8105900 con 0x7f04d003e470 2024-05-03T18:37:25.497 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:24.960+0000 7f04e57fa700 1 -- 172.21.15.32:0/3740738237 <== mgr.14120 v2:172.21.15.32:6800/2019702098 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f04e8105900 con 0x7f04d003e470 2024-05-03T18:37:25.497 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:24.962+0000 7f04ee4a6700 1 -- 172.21.15.32:0/3740738237 >> [v2:172.21.15.32:6800/2019702098,v1:172.21.15.32:6801/2019702098] conn(0x7f04d003e470 msgr2=0x7f04d0040930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:37:25.497 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:24.962+0000 7f04ee4a6700 1 --2- 172.21.15.32:0/3740738237 >> [v2:172.21.15.32:6800/2019702098,v1:172.21.15.32:6801/2019702098] conn(0x7f04d003e470 0x7f04d0040930 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f04d8006fd0 tx=0x7f04d8006e40 comp rx=0 tx=0).stop 2024-05-03T18:37:25.497 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:24.962+0000 7f04ee4a6700 1 -- 172.21.15.32:0/3740738237 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04e807ee30 msgr2=0x7f04e819f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:37:25.497 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:24.962+0000 7f04ee4a6700 1 --2- 172.21.15.32:0/3740738237 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04e807ee30 0x7f04e819f100 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f04dc011ac0 tx=0x7f04dc011ba0 comp rx=0 tx=0).stop 2024-05-03T18:37:25.497 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:24.962+0000 7f04ee4a6700 1 -- 172.21.15.32:0/3740738237 shutdown_connections 2024-05-03T18:37:25.497 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:24.962+0000 7f04ee4a6700 1 --2- 172.21.15.32:0/3740738237 >> [v2:172.21.15.32:6800/2019702098,v1:172.21.15.32:6801/2019702098] conn(0x7f04d003e470 0x7f04d0040930 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:25.498 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:24.962+0000 7f04ee4a6700 1 --2- 172.21.15.32:0/3740738237 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04e807ee30 0x7f04e819f100 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:25.498 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:24.962+0000 7f04ee4a6700 1 -- 172.21.15.32:0/3740738237 >> 172.21.15.32:0/3740738237 conn(0x7f04e8104650 msgr2=0x7f04e81051f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:37:25.498 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:24.962+0000 7f04ee4a6700 1 -- 172.21.15.32:0/3740738237 shutdown_connections 2024-05-03T18:37:25.498 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:24.962+0000 7f04ee4a6700 1 -- 172.21.15.32:0/3740738237 wait complete. 2024-05-03T18:37:25.498 INFO:teuthology.orchestra.run.smithi032.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-05-03T18:37:25.498 INFO:teuthology.orchestra.run.smithi032.stdout:Adding key to root@localhost authorized_keys... 2024-05-03T18:37:25.498 INFO:teuthology.orchestra.run.smithi032.stdout:Adding host smithi032... 2024-05-03T18:37:25.878 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:37:25 smithi032 ceph-mon[29763]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:37:27.018 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:37:26 smithi032 ceph-mon[29763]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi032", "addr": "172.21.15.32", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:37:28.067 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:37:27 smithi032 ceph-mon[29763]: Deploying cephadm binary to smithi032 2024-05-03T18:37:30.940 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout Added host 'smithi032' with addr '172.21.15.32' 2024-05-03T18:37:30.940 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:25.877+0000 7f11d782d700 1 Processor -- start 2024-05-03T18:37:30.940 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:25.877+0000 7f11d782d700 1 -- start start 2024-05-03T18:37:30.941 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:25.877+0000 7f11d782d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f11d007ee10 0x7f11d007d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:30.941 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:25.877+0000 7f11d782d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f11d007d850 con 0x7f11d007ee10 2024-05-03T18:37:30.941 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:25.878+0000 7f11d55c9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f11d007ee10 0x7f11d007d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:30.941 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:25.878+0000 7f11d55c9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f11d007ee10 0x7f11d007d310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53106/0 (socket says 172.21.15.32:53106) 2024-05-03T18:37:30.941 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:25.878+0000 7f11d55c9700 1 -- 172.21.15.32:0/2350249674 learned_addr learned my addr 172.21.15.32:0/2350249674 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:37:30.941 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:25.878+0000 7f11d55c9700 1 -- 172.21.15.32:0/2350249674 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11d007d9d0 con 0x7f11d007ee10 2024-05-03T18:37:30.941 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:25.878+0000 7f11d55c9700 1 --2- 172.21.15.32:0/2350249674 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f11d007ee10 0x7f11d007d310 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f11c8009e80 tx=0x7f11c800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31ecc9bb99adf935 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:37:30.941 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:25.878+0000 7f11cffff700 1 -- 172.21.15.32:0/2350249674 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f11c8017070 con 0x7f11d007ee10 2024-05-03T18:37:30.941 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:25.878+0000 7f11cffff700 1 -- 172.21.15.32:0/2350249674 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f11c800b8c0 con 0x7f11d007ee10 2024-05-03T18:37:30.941 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:25.878+0000 7f11cffff700 1 -- 172.21.15.32:0/2350249674 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f11c8004490 con 0x7f11d007ee10 2024-05-03T18:37:30.941 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:25.879+0000 7f11d782d700 1 -- 172.21.15.32:0/2350249674 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f11d007ee10 msgr2=0x7f11d007d310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:37:30.941 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:25.879+0000 7f11d782d700 1 --2- 172.21.15.32:0/2350249674 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f11d007ee10 0x7f11d007d310 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f11c8009e80 tx=0x7f11c800b330 comp rx=0 tx=0).stop 2024-05-03T18:37:30.941 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:25.879+0000 7f11d782d700 1 -- 172.21.15.32:0/2350249674 shutdown_connections 2024-05-03T18:37:30.941 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:25.879+0000 7f11d782d700 1 --2- 172.21.15.32:0/2350249674 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f11d007ee10 0x7f11d007d310 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:30.942 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:25.879+0000 7f11d782d700 1 -- 172.21.15.32:0/2350249674 >> 172.21.15.32:0/2350249674 conn(0x7f11d0104650 msgr2=0x7f11d0106a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:37:30.942 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:25.879+0000 7f11d782d700 1 -- 172.21.15.32:0/2350249674 shutdown_connections 2024-05-03T18:37:30.942 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:25.879+0000 7f11d782d700 1 -- 172.21.15.32:0/2350249674 wait complete. 2024-05-03T18:37:30.942 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:25.880+0000 7f11d782d700 1 Processor -- start 2024-05-03T18:37:30.942 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:25.880+0000 7f11d782d700 1 -- start start 2024-05-03T18:37:30.942 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:25.880+0000 7f11d782d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f11d007ee10 0x7f11d019f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:30.942 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:25.880+0000 7f11d782d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f11d010f560 con 0x7f11d007ee10 2024-05-03T18:37:30.942 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:25.880+0000 7f11d55c9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f11d007ee10 0x7f11d019f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:30.942 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:25.880+0000 7f11d55c9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f11d007ee10 0x7f11d019f0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53112/0 (socket says 172.21.15.32:53112) 2024-05-03T18:37:30.942 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:25.880+0000 7f11d55c9700 1 -- 172.21.15.32:0/1921409125 learned_addr learned my addr 172.21.15.32:0/1921409125 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:37:30.943 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:25.880+0000 7f11d55c9700 1 -- 172.21.15.32:0/1921409125 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11d019f5f0 con 0x7f11d007ee10 2024-05-03T18:37:30.943 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:25.880+0000 7f11d55c9700 1 --2- 172.21.15.32:0/1921409125 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f11d007ee10 0x7f11d019f0b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f11c800b7b0 tx=0x7f11c800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:37:30.943 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:25.880+0000 7f11ce7fc700 1 -- 172.21.15.32:0/1921409125 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f11c8020020 con 0x7f11d007ee10 2024-05-03T18:37:30.943 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:25.881+0000 7f11ce7fc700 1 -- 172.21.15.32:0/1921409125 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f11c8012430 con 0x7f11d007ee10 2024-05-03T18:37:30.943 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:25.881+0000 7f11d782d700 1 -- 172.21.15.32:0/1921409125 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11d019f880 con 0x7f11d007ee10 2024-05-03T18:37:30.943 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:25.881+0000 7f11ce7fc700 1 -- 172.21.15.32:0/1921409125 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f11c8017040 con 0x7f11d007ee10 2024-05-03T18:37:30.943 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:25.881+0000 7f11d782d700 1 -- 172.21.15.32:0/1921409125 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11d01a2570 con 0x7f11d007ee10 2024-05-03T18:37:30.943 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:25.881+0000 7f11ce7fc700 1 -- 172.21.15.32:0/1921409125 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f11c8023020 con 0x7f11d007ee10 2024-05-03T18:37:30.943 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:25.882+0000 7f11d782d700 1 -- 172.21.15.32:0/1921409125 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11d0064b00 con 0x7f11d007ee10 2024-05-03T18:37:30.943 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:25.882+0000 7f11ce7fc700 1 --2- 172.21.15.32:0/1921409125 >> [v2:172.21.15.32:6800/2019702098,v1:172.21.15.32:6801/2019702098] conn(0x7f11b803e470 0x7f11b8040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:30.943 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:25.882+0000 7f11ce7fc700 1 -- 172.21.15.32:0/1921409125 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f11c800f050 con 0x7f11d007ee10 2024-05-03T18:37:30.944 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:25.882+0000 7f11d4dc8700 1 --2- 172.21.15.32:0/1921409125 >> [v2:172.21.15.32:6800/2019702098,v1:172.21.15.32:6801/2019702098] conn(0x7f11b803e470 0x7f11b8040930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:30.944 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:25.882+0000 7f11d4dc8700 1 --2- 172.21.15.32:0/1921409125 >> [v2:172.21.15.32:6800/2019702098,v1:172.21.15.32:6801/2019702098] conn(0x7f11b803e470 0x7f11b8040930 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f11c4006fd0 tx=0x7f11c4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:37:30.944 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:25.887+0000 7f11ce7fc700 1 -- 172.21.15.32:0/1921409125 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f11c802a020 con 0x7f11d007ee10 2024-05-03T18:37:30.944 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:26.104+0000 7f11d782d700 1 -- 172.21.15.32:0/1921409125 --> [v2:172.21.15.32:6800/2019702098,v1:172.21.15.32:6801/2019702098] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi032", "addr": "172.21.15.32", "target": ["mon-mgr", ""]}) -- 0x7f11d01058c0 con 0x7f11b803e470 2024-05-03T18:37:30.944 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:30.362+0000 7f11ce7fc700 1 -- 172.21.15.32:0/1921409125 <== mgr.14120 v2:172.21.15.32:6800/2019702098 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f11d01058c0 con 0x7f11b803e470 2024-05-03T18:37:30.944 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:30.365+0000 7f11d782d700 1 -- 172.21.15.32:0/1921409125 >> [v2:172.21.15.32:6800/2019702098,v1:172.21.15.32:6801/2019702098] conn(0x7f11b803e470 msgr2=0x7f11b8040930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:37:30.944 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:30.365+0000 7f11d782d700 1 --2- 172.21.15.32:0/1921409125 >> [v2:172.21.15.32:6800/2019702098,v1:172.21.15.32:6801/2019702098] conn(0x7f11b803e470 0x7f11b8040930 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f11c4006fd0 tx=0x7f11c4006e40 comp rx=0 tx=0).stop 2024-05-03T18:37:30.944 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:30.365+0000 7f11d782d700 1 -- 172.21.15.32:0/1921409125 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f11d007ee10 msgr2=0x7f11d019f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:37:30.944 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:30.365+0000 7f11d782d700 1 --2- 172.21.15.32:0/1921409125 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f11d007ee10 0x7f11d019f0b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f11c800b7b0 tx=0x7f11c800be20 comp rx=0 tx=0).stop 2024-05-03T18:37:30.944 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:30.365+0000 7f11d782d700 1 -- 172.21.15.32:0/1921409125 shutdown_connections 2024-05-03T18:37:30.944 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:30.365+0000 7f11d782d700 1 --2- 172.21.15.32:0/1921409125 >> [v2:172.21.15.32:6800/2019702098,v1:172.21.15.32:6801/2019702098] conn(0x7f11b803e470 0x7f11b8040930 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:30.944 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:30.365+0000 7f11d782d700 1 --2- 172.21.15.32:0/1921409125 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f11d007ee10 0x7f11d019f0b0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:30.944 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:30.365+0000 7f11d782d700 1 -- 172.21.15.32:0/1921409125 >> 172.21.15.32:0/1921409125 conn(0x7f11d0104650 msgr2=0x7f11d01051b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:37:30.944 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:30.365+0000 7f11d782d700 1 -- 172.21.15.32:0/1921409125 shutdown_connections 2024-05-03T18:37:30.945 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:30.365+0000 7f11d782d700 1 -- 172.21.15.32:0/1921409125 wait complete. 2024-05-03T18:37:30.945 INFO:teuthology.orchestra.run.smithi032.stdout:Deploying unmanaged mon service... 2024-05-03T18:37:31.505 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:37:31 smithi032 ceph-mon[29763]: from='mgr.14120 172.21.15.32:0/2820864439' entity='mgr.x' 2024-05-03T18:37:31.506 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:37:31 smithi032 ceph-mon[29763]: Added host smithi032 2024-05-03T18:37:31.506 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:37:31 smithi032 ceph-mon[29763]: from='mgr.14120 172.21.15.32:0/2820864439' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:37:32.971 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:37:32 smithi032 ceph-mon[29763]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:37:32.971 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:37:32 smithi032 ceph-mon[29763]: Saving service mon spec with placement count:5 2024-05-03T18:37:32.971 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:37:32 smithi032 ceph-mon[29763]: from='mgr.14120 172.21.15.32:0/2820864439' entity='mgr.x' 2024-05-03T18:37:32.979 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-05-03T18:37:32.979 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:31.357+0000 7f2a2602c700 1 Processor -- start 2024-05-03T18:37:32.979 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:31.357+0000 7f2a2602c700 1 -- start start 2024-05-03T18:37:32.979 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:31.357+0000 7f2a2602c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a20107850 0x7f2a20107c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:32.979 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:31.357+0000 7f2a2602c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f2a2007ce60 con 0x7f2a20107850 2024-05-03T18:37:32.979 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:31.357+0000 7f2a1f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a20107850 0x7f2a20107c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:32.979 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:31.357+0000 7f2a1f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a20107850 0x7f2a20107c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53126/0 (socket says 172.21.15.32:53126) 2024-05-03T18:37:32.980 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:31.357+0000 7f2a1f7fe700 1 -- 172.21.15.32:0/1929089382 learned_addr learned my addr 172.21.15.32:0/1929089382 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:37:32.980 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:31.357+0000 7f2a1f7fe700 1 -- 172.21.15.32:0/1929089382 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a201081b0 con 0x7f2a20107850 2024-05-03T18:37:32.980 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:31.357+0000 7f2a1f7fe700 1 --2- 172.21.15.32:0/1929089382 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a20107850 0x7f2a20107c70 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f2a18009e80 tx=0x7f2a1800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f4dadf14c7844aa server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:37:32.980 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:31.358+0000 7f2a1e7fc700 1 -- 172.21.15.32:0/1929089382 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2a18017070 con 0x7f2a20107850 2024-05-03T18:37:32.980 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:31.358+0000 7f2a1e7fc700 1 -- 172.21.15.32:0/1929089382 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f2a1800b8c0 con 0x7f2a20107850 2024-05-03T18:37:32.980 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:31.358+0000 7f2a1e7fc700 1 -- 172.21.15.32:0/1929089382 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2a18004490 con 0x7f2a20107850 2024-05-03T18:37:32.980 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:31.358+0000 7f2a2602c700 1 -- 172.21.15.32:0/1929089382 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a20107850 msgr2=0x7f2a20107c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:37:32.980 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:31.358+0000 7f2a2602c700 1 --2- 172.21.15.32:0/1929089382 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a20107850 0x7f2a20107c70 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f2a18009e80 tx=0x7f2a1800b330 comp rx=0 tx=0).stop 2024-05-03T18:37:32.980 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:31.358+0000 7f2a2602c700 1 -- 172.21.15.32:0/1929089382 shutdown_connections 2024-05-03T18:37:32.980 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:31.358+0000 7f2a2602c700 1 --2- 172.21.15.32:0/1929089382 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a20107850 0x7f2a20107c70 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:32.980 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:31.358+0000 7f2a2602c700 1 -- 172.21.15.32:0/1929089382 >> 172.21.15.32:0/1929089382 conn(0x7f2a20103450 msgr2=0x7f2a20105870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:37:32.980 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:31.358+0000 7f2a2602c700 1 -- 172.21.15.32:0/1929089382 shutdown_connections 2024-05-03T18:37:32.980 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:31.358+0000 7f2a2602c700 1 -- 172.21.15.32:0/1929089382 wait complete. 2024-05-03T18:37:32.980 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:31.359+0000 7f2a2602c700 1 Processor -- start 2024-05-03T18:37:32.981 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:31.359+0000 7f2a2602c700 1 -- start start 2024-05-03T18:37:32.981 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:31.359+0000 7f2a2602c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a20107850 0x7f2a2019f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:32.981 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:31.359+0000 7f2a2602c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f2a201091e0 con 0x7f2a20107850 2024-05-03T18:37:32.981 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:31.359+0000 7f2a1f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a20107850 0x7f2a2019f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:32.981 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:31.359+0000 7f2a1f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a20107850 0x7f2a2019f0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53140/0 (socket says 172.21.15.32:53140) 2024-05-03T18:37:32.981 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:31.359+0000 7f2a1f7fe700 1 -- 172.21.15.32:0/3211552348 learned_addr learned my addr 172.21.15.32:0/3211552348 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:37:32.981 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:31.359+0000 7f2a1f7fe700 1 -- 172.21.15.32:0/3211552348 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a2019f630 con 0x7f2a20107850 2024-05-03T18:37:32.981 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:31.360+0000 7f2a1f7fe700 1 --2- 172.21.15.32:0/3211552348 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a20107850 0x7f2a2019f0f0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f2a1800b7b0 tx=0x7f2a1800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:37:32.981 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:31.360+0000 7f2a1cff9700 1 -- 172.21.15.32:0/3211552348 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2a18020020 con 0x7f2a20107850 2024-05-03T18:37:32.981 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:31.360+0000 7f2a1cff9700 1 -- 172.21.15.32:0/3211552348 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f2a18012430 con 0x7f2a20107850 2024-05-03T18:37:32.981 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:31.360+0000 7f2a2602c700 1 -- 172.21.15.32:0/3211552348 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a2019f8c0 con 0x7f2a20107850 2024-05-03T18:37:32.981 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:31.360+0000 7f2a1cff9700 1 -- 172.21.15.32:0/3211552348 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2a18017040 con 0x7f2a20107850 2024-05-03T18:37:32.981 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:31.360+0000 7f2a2602c700 1 -- 172.21.15.32:0/3211552348 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a201a25b0 con 0x7f2a20107850 2024-05-03T18:37:32.981 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:31.361+0000 7f2a2602c700 1 -- 172.21.15.32:0/3211552348 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a20064b00 con 0x7f2a20107850 2024-05-03T18:37:32.981 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:31.361+0000 7f2a1cff9700 1 -- 172.21.15.32:0/3211552348 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f2a18023020 con 0x7f2a20107850 2024-05-03T18:37:32.982 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:31.362+0000 7f2a1cff9700 1 --2- 172.21.15.32:0/3211552348 >> [v2:172.21.15.32:6800/2019702098,v1:172.21.15.32:6801/2019702098] conn(0x7f2a0c03e420 0x7f2a0c0408e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:32.982 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:31.362+0000 7f2a1cff9700 1 -- 172.21.15.32:0/3211552348 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f2a1800f050 con 0x7f2a20107850 2024-05-03T18:37:32.982 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:31.362+0000 7f2a1effd700 1 --2- 172.21.15.32:0/3211552348 >> [v2:172.21.15.32:6800/2019702098,v1:172.21.15.32:6801/2019702098] conn(0x7f2a0c03e420 0x7f2a0c0408e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:32.982 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:31.362+0000 7f2a1effd700 1 --2- 172.21.15.32:0/3211552348 >> [v2:172.21.15.32:6800/2019702098,v1:172.21.15.32:6801/2019702098] conn(0x7f2a0c03e420 0x7f2a0c0408e0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f2a08006fd0 tx=0x7f2a08006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:37:32.982 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:31.369+0000 7f2a1cff9700 1 -- 172.21.15.32:0/3211552348 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f2a1802a020 con 0x7f2a20107850 2024-05-03T18:37:32.982 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:31.606+0000 7f2a2602c700 1 -- 172.21.15.32:0/3211552348 --> [v2:172.21.15.32:6800/2019702098,v1:172.21.15.32:6801/2019702098] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f2a2002e8c0 con 0x7f2a0c03e420 2024-05-03T18:37:32.982 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:31.611+0000 7f2a1cff9700 1 -- 172.21.15.32:0/3211552348 <== mgr.14120 v2:172.21.15.32:6800/2019702098 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f2a2002e8c0 con 0x7f2a0c03e420 2024-05-03T18:37:32.982 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:31.613+0000 7f2a2602c700 1 -- 172.21.15.32:0/3211552348 >> [v2:172.21.15.32:6800/2019702098,v1:172.21.15.32:6801/2019702098] conn(0x7f2a0c03e420 msgr2=0x7f2a0c0408e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:37:32.982 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:31.613+0000 7f2a2602c700 1 --2- 172.21.15.32:0/3211552348 >> [v2:172.21.15.32:6800/2019702098,v1:172.21.15.32:6801/2019702098] conn(0x7f2a0c03e420 0x7f2a0c0408e0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f2a08006fd0 tx=0x7f2a08006e40 comp rx=0 tx=0).stop 2024-05-03T18:37:32.982 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:31.613+0000 7f2a2602c700 1 -- 172.21.15.32:0/3211552348 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a20107850 msgr2=0x7f2a2019f0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:37:32.982 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:31.613+0000 7f2a2602c700 1 --2- 172.21.15.32:0/3211552348 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a20107850 0x7f2a2019f0f0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f2a1800b7b0 tx=0x7f2a1800be20 comp rx=0 tx=0).stop 2024-05-03T18:37:32.982 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:31.613+0000 7f2a2602c700 1 -- 172.21.15.32:0/3211552348 shutdown_connections 2024-05-03T18:37:32.982 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:31.613+0000 7f2a2602c700 1 --2- 172.21.15.32:0/3211552348 >> [v2:172.21.15.32:6800/2019702098,v1:172.21.15.32:6801/2019702098] conn(0x7f2a0c03e420 0x7f2a0c0408e0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:32.983 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:31.613+0000 7f2a2602c700 1 --2- 172.21.15.32:0/3211552348 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a20107850 0x7f2a2019f0f0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:32.983 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:31.613+0000 7f2a2602c700 1 -- 172.21.15.32:0/3211552348 >> 172.21.15.32:0/3211552348 conn(0x7f2a20103450 msgr2=0x7f2a20195d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:37:32.983 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:31.613+0000 7f2a2602c700 1 -- 172.21.15.32:0/3211552348 shutdown_connections 2024-05-03T18:37:32.983 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:31.613+0000 7f2a2602c700 1 -- 172.21.15.32:0/3211552348 wait complete. 2024-05-03T18:37:32.983 INFO:teuthology.orchestra.run.smithi032.stdout:Deploying unmanaged mgr service... 2024-05-03T18:37:34.052 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:37:33 smithi032 ceph-mon[29763]: from='mgr.14120 172.21.15.32:0/2820864439' entity='mgr.x' 2024-05-03T18:37:34.053 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:37:33 smithi032 ceph-mon[29763]: from='mgr.14120 172.21.15.32:0/2820864439' entity='mgr.x' 2024-05-03T18:37:34.330 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-05-03T18:37:34.330 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:33.463+0000 7f3d705a8700 1 Processor -- start 2024-05-03T18:37:34.330 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:33.463+0000 7f3d705a8700 1 -- start start 2024-05-03T18:37:34.330 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:33.463+0000 7f3d705a8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3d68107e90 0x7f3d681082b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:34.330 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:33.463+0000 7f3d705a8700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f3d681087f0 con 0x7f3d68107e90 2024-05-03T18:37:34.330 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:33.463+0000 7f3d6e344700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3d68107e90 0x7f3d681082b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:34.331 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:33.464+0000 7f3d6e344700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3d68107e90 0x7f3d681082b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53152/0 (socket says 172.21.15.32:53152) 2024-05-03T18:37:34.331 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:33.464+0000 7f3d6e344700 1 -- 172.21.15.32:0/816776490 learned_addr learned my addr 172.21.15.32:0/816776490 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:37:34.331 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:33.464+0000 7f3d6e344700 1 -- 172.21.15.32:0/816776490 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d68108ff0 con 0x7f3d68107e90 2024-05-03T18:37:34.331 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:33.464+0000 7f3d6e344700 1 --2- 172.21.15.32:0/816776490 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3d68107e90 0x7f3d681082b0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f3d580098a0 tx=0x7f3d58009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=317a5ca5eac87009 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:37:34.331 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:33.464+0000 7f3d6d342700 1 -- 172.21.15.32:0/816776490 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3d58004160 con 0x7f3d68107e90 2024-05-03T18:37:34.331 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:33.464+0000 7f3d6d342700 1 -- 172.21.15.32:0/816776490 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d58004300 con 0x7f3d68107e90 2024-05-03T18:37:34.331 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:33.464+0000 7f3d6d342700 1 -- 172.21.15.32:0/816776490 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3d58011a20 con 0x7f3d68107e90 2024-05-03T18:37:34.331 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:33.465+0000 7f3d705a8700 1 -- 172.21.15.32:0/816776490 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3d68107e90 msgr2=0x7f3d681082b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:37:34.331 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:33.465+0000 7f3d705a8700 1 --2- 172.21.15.32:0/816776490 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3d68107e90 0x7f3d681082b0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f3d580098a0 tx=0x7f3d58009bb0 comp rx=0 tx=0).stop 2024-05-03T18:37:34.331 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:33.465+0000 7f3d705a8700 1 -- 172.21.15.32:0/816776490 shutdown_connections 2024-05-03T18:37:34.331 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:33.465+0000 7f3d705a8700 1 --2- 172.21.15.32:0/816776490 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3d68107e90 0x7f3d681082b0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:34.331 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:33.465+0000 7f3d705a8700 1 -- 172.21.15.32:0/816776490 >> 172.21.15.32:0/816776490 conn(0x7f3d68103450 msgr2=0x7f3d68105870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:37:34.331 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:33.465+0000 7f3d705a8700 1 -- 172.21.15.32:0/816776490 shutdown_connections 2024-05-03T18:37:34.331 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:33.465+0000 7f3d705a8700 1 -- 172.21.15.32:0/816776490 wait complete. 2024-05-03T18:37:34.332 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:33.465+0000 7f3d705a8700 1 Processor -- start 2024-05-03T18:37:34.332 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:33.466+0000 7f3d705a8700 1 -- start start 2024-05-03T18:37:34.332 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:33.466+0000 7f3d705a8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3d68107e90 0x7f3d6819cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:34.332 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:33.466+0000 7f3d705a8700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f3d6810c430 con 0x7f3d68107e90 2024-05-03T18:37:34.332 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:33.466+0000 7f3d6e344700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3d68107e90 0x7f3d6819cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:34.332 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:33.466+0000 7f3d6e344700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3d68107e90 0x7f3d6819cea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53168/0 (socket says 172.21.15.32:53168) 2024-05-03T18:37:34.332 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:33.466+0000 7f3d6e344700 1 -- 172.21.15.32:0/760523615 learned_addr learned my addr 172.21.15.32:0/760523615 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:37:34.332 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:33.466+0000 7f3d6e344700 1 -- 172.21.15.32:0/760523615 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d6819d3e0 con 0x7f3d68107e90 2024-05-03T18:37:34.332 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:33.466+0000 7f3d6e344700 1 --2- 172.21.15.32:0/760523615 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3d68107e90 0x7f3d6819cea0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f3d580116e0 tx=0x7f3d58011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:37:34.332 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:33.466+0000 7f3d5f7fe700 1 -- 172.21.15.32:0/760523615 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3d58003a60 con 0x7f3d68107e90 2024-05-03T18:37:34.332 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:33.466+0000 7f3d705a8700 1 -- 172.21.15.32:0/760523615 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d6819d670 con 0x7f3d68107e90 2024-05-03T18:37:34.332 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:33.466+0000 7f3d5f7fe700 1 -- 172.21.15.32:0/760523615 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d5801b8f0 con 0x7f3d68107e90 2024-05-03T18:37:34.332 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:33.467+0000 7f3d5f7fe700 1 -- 172.21.15.32:0/760523615 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3d58019860 con 0x7f3d68107e90 2024-05-03T18:37:34.332 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:33.467+0000 7f3d705a8700 1 -- 172.21.15.32:0/760523615 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d681a0360 con 0x7f3d68107e90 2024-05-03T18:37:34.333 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:33.468+0000 7f3d5f7fe700 1 -- 172.21.15.32:0/760523615 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f3d58019b80 con 0x7f3d68107e90 2024-05-03T18:37:34.333 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:33.468+0000 7f3d5f7fe700 1 --2- 172.21.15.32:0/760523615 >> [v2:172.21.15.32:6800/2019702098,v1:172.21.15.32:6801/2019702098] conn(0x7f3d5403e420 0x7f3d540408e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:34.333 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:33.468+0000 7f3d705a8700 1 -- 172.21.15.32:0/760523615 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d6810c5b0 con 0x7f3d68107e90 2024-05-03T18:37:34.333 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:33.468+0000 7f3d5f7fe700 1 -- 172.21.15.32:0/760523615 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f3d58053c70 con 0x7f3d68107e90 2024-05-03T18:37:34.333 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:33.468+0000 7f3d6db43700 1 --2- 172.21.15.32:0/760523615 >> [v2:172.21.15.32:6800/2019702098,v1:172.21.15.32:6801/2019702098] conn(0x7f3d5403e420 0x7f3d540408e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:34.333 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:33.469+0000 7f3d6db43700 1 --2- 172.21.15.32:0/760523615 >> [v2:172.21.15.32:6800/2019702098,v1:172.21.15.32:6801/2019702098] conn(0x7f3d5403e420 0x7f3d540408e0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f3d64006fd0 tx=0x7f3d64006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:37:34.333 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:33.473+0000 7f3d5f7fe700 1 -- 172.21.15.32:0/760523615 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f3d5801f030 con 0x7f3d68107e90 2024-05-03T18:37:34.333 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:33.750+0000 7f3d705a8700 1 -- 172.21.15.32:0/760523615 --> [v2:172.21.15.32:6800/2019702098,v1:172.21.15.32:6801/2019702098] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f3d6802e7c0 con 0x7f3d5403e420 2024-05-03T18:37:34.333 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:33.755+0000 7f3d5f7fe700 1 -- 172.21.15.32:0/760523615 <== mgr.14120 v2:172.21.15.32:6800/2019702098 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f3d6802e7c0 con 0x7f3d5403e420 2024-05-03T18:37:34.333 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:33.758+0000 7f3d705a8700 1 -- 172.21.15.32:0/760523615 >> [v2:172.21.15.32:6800/2019702098,v1:172.21.15.32:6801/2019702098] conn(0x7f3d5403e420 msgr2=0x7f3d540408e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:37:34.333 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:33.758+0000 7f3d705a8700 1 --2- 172.21.15.32:0/760523615 >> [v2:172.21.15.32:6800/2019702098,v1:172.21.15.32:6801/2019702098] conn(0x7f3d5403e420 0x7f3d540408e0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f3d64006fd0 tx=0x7f3d64006e40 comp rx=0 tx=0).stop 2024-05-03T18:37:34.333 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:33.758+0000 7f3d705a8700 1 -- 172.21.15.32:0/760523615 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3d68107e90 msgr2=0x7f3d6819cea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:37:34.333 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:33.758+0000 7f3d705a8700 1 --2- 172.21.15.32:0/760523615 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3d68107e90 0x7f3d6819cea0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f3d580116e0 tx=0x7f3d58011aa0 comp rx=0 tx=0).stop 2024-05-03T18:37:34.333 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:33.758+0000 7f3d705a8700 1 -- 172.21.15.32:0/760523615 shutdown_connections 2024-05-03T18:37:34.334 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:33.758+0000 7f3d705a8700 1 --2- 172.21.15.32:0/760523615 >> [v2:172.21.15.32:6800/2019702098,v1:172.21.15.32:6801/2019702098] conn(0x7f3d5403e420 0x7f3d540408e0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:34.334 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:33.758+0000 7f3d705a8700 1 --2- 172.21.15.32:0/760523615 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3d68107e90 0x7f3d6819cea0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:34.334 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:33.758+0000 7f3d705a8700 1 -- 172.21.15.32:0/760523615 >> 172.21.15.32:0/760523615 conn(0x7f3d68103450 msgr2=0x7f3d681042a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:37:34.334 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:33.758+0000 7f3d705a8700 1 -- 172.21.15.32:0/760523615 shutdown_connections 2024-05-03T18:37:34.334 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:33.758+0000 7f3d705a8700 1 -- 172.21.15.32:0/760523615 wait complete. 2024-05-03T18:37:34.978 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:37:34 smithi032 ceph-mon[29763]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:37:34.978 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:37:34 smithi032 ceph-mon[29763]: Saving service mgr spec with placement count:2 2024-05-03T18:37:34.979 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:37:34 smithi032 ceph-mon[29763]: from='mgr.14120 172.21.15.32:0/2820864439' entity='mgr.x' 2024-05-03T18:37:35.646 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:34.748+0000 7f4523bca700 1 Processor -- start 2024-05-03T18:37:35.646 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:34.748+0000 7f4523bca700 1 -- start start 2024-05-03T18:37:35.646 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:34.748+0000 7f4523bca700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f451c109a00 0x7f451c10bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:35.646 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:34.748+0000 7f4523bca700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f451c07cec0 con 0x7f451c109a00 2024-05-03T18:37:35.646 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:34.748+0000 7f4521966700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f451c109a00 0x7f451c10bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:35.646 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:34.748+0000 7f4521966700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f451c109a00 0x7f451c10bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38686/0 (socket says 172.21.15.32:38686) 2024-05-03T18:37:35.646 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:34.748+0000 7f4521966700 1 -- 172.21.15.32:0/1833302632 learned_addr learned my addr 172.21.15.32:0/1833302632 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:37:35.646 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:34.749+0000 7f4521966700 1 -- 172.21.15.32:0/1833302632 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f451c10c330 con 0x7f451c109a00 2024-05-03T18:37:35.646 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:34.749+0000 7f4521966700 1 --2- 172.21.15.32:0/1833302632 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f451c109a00 0x7f451c10bdf0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f45180098a0 tx=0x7f4518009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a373c9b6ffb3a8c0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:37:35.647 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:34.749+0000 7f4520964700 1 -- 172.21.15.32:0/1833302632 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4518004160 con 0x7f451c109a00 2024-05-03T18:37:35.647 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:34.749+0000 7f4520964700 1 -- 172.21.15.32:0/1833302632 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4518004300 con 0x7f451c109a00 2024-05-03T18:37:35.647 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:34.749+0000 7f4520964700 1 -- 172.21.15.32:0/1833302632 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4518011a20 con 0x7f451c109a00 2024-05-03T18:37:35.647 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:34.750+0000 7f4523bca700 1 -- 172.21.15.32:0/1833302632 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f451c109a00 msgr2=0x7f451c10bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:37:35.647 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:34.750+0000 7f4523bca700 1 --2- 172.21.15.32:0/1833302632 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f451c109a00 0x7f451c10bdf0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f45180098a0 tx=0x7f4518009bb0 comp rx=0 tx=0).stop 2024-05-03T18:37:35.647 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:34.750+0000 7f4523bca700 1 -- 172.21.15.32:0/1833302632 shutdown_connections 2024-05-03T18:37:35.647 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:34.750+0000 7f4523bca700 1 --2- 172.21.15.32:0/1833302632 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f451c109a00 0x7f451c10bdf0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:35.647 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:34.750+0000 7f4523bca700 1 -- 172.21.15.32:0/1833302632 >> 172.21.15.32:0/1833302632 conn(0x7f451c1034d0 msgr2=0x7f451c1058f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:37:35.647 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:34.750+0000 7f4523bca700 1 -- 172.21.15.32:0/1833302632 shutdown_connections 2024-05-03T18:37:35.647 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:34.751+0000 7f4523bca700 1 -- 172.21.15.32:0/1833302632 wait complete. 2024-05-03T18:37:35.647 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:34.751+0000 7f4523bca700 1 Processor -- start 2024-05-03T18:37:35.647 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:34.751+0000 7f4523bca700 1 -- start start 2024-05-03T18:37:35.647 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:34.751+0000 7f4523bca700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f451c109a00 0x7f451c19f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:35.647 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:34.751+0000 7f4523bca700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f451c10d340 con 0x7f451c109a00 2024-05-03T18:37:35.648 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:34.751+0000 7f4521966700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f451c109a00 0x7f451c19f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:35.648 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:34.752+0000 7f4521966700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f451c109a00 0x7f451c19f0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38700/0 (socket says 172.21.15.32:38700) 2024-05-03T18:37:35.648 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:34.752+0000 7f4521966700 1 -- 172.21.15.32:0/4279498952 learned_addr learned my addr 172.21.15.32:0/4279498952 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:37:35.648 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:34.752+0000 7f4521966700 1 -- 172.21.15.32:0/4279498952 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f451c19f630 con 0x7f451c109a00 2024-05-03T18:37:35.648 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:34.752+0000 7f4521966700 1 --2- 172.21.15.32:0/4279498952 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f451c109a00 0x7f451c19f0f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f45180116e0 tx=0x7f4518011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:37:35.648 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:34.752+0000 7f4512ffd700 1 -- 172.21.15.32:0/4279498952 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4518003b20 con 0x7f451c109a00 2024-05-03T18:37:35.648 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:34.752+0000 7f4523bca700 1 -- 172.21.15.32:0/4279498952 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f451c19f8c0 con 0x7f451c109a00 2024-05-03T18:37:35.648 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:34.752+0000 7f4523bca700 1 -- 172.21.15.32:0/4279498952 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f451c1a25b0 con 0x7f451c109a00 2024-05-03T18:37:35.648 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:34.752+0000 7f4512ffd700 1 -- 172.21.15.32:0/4279498952 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f451801b8f0 con 0x7f451c109a00 2024-05-03T18:37:35.648 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:34.752+0000 7f4512ffd700 1 -- 172.21.15.32:0/4279498952 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f45180238c0 con 0x7f451c109a00 2024-05-03T18:37:35.648 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:34.754+0000 7f4512ffd700 1 -- 172.21.15.32:0/4279498952 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f4518023b40 con 0x7f451c109a00 2024-05-03T18:37:35.648 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:34.754+0000 7f4523bca700 1 -- 172.21.15.32:0/4279498952 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4500005360 con 0x7f451c109a00 2024-05-03T18:37:35.648 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:34.754+0000 7f4512ffd700 1 --2- 172.21.15.32:0/4279498952 >> [v2:172.21.15.32:6800/2019702098,v1:172.21.15.32:6801/2019702098] conn(0x7f450803e4d0 0x7f4508040990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:35.648 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:34.754+0000 7f4512ffd700 1 -- 172.21.15.32:0/4279498952 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f4518053fd0 con 0x7f451c109a00 2024-05-03T18:37:35.648 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:34.754+0000 7f4521165700 1 --2- 172.21.15.32:0/4279498952 >> [v2:172.21.15.32:6800/2019702098,v1:172.21.15.32:6801/2019702098] conn(0x7f450803e4d0 0x7f4508040990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:35.649 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:34.755+0000 7f4521165700 1 --2- 172.21.15.32:0/4279498952 >> [v2:172.21.15.32:6800/2019702098,v1:172.21.15.32:6801/2019702098] conn(0x7f450803e4d0 0x7f4508040990 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f450c006fd0 tx=0x7f450c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:37:35.649 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:34.761+0000 7f4512ffd700 1 -- 172.21.15.32:0/4279498952 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f451801f030 con 0x7f451c109a00 2024-05-03T18:37:35.649 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:34.977+0000 7f4523bca700 1 -- 172.21.15.32:0/4279498952 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f4500005650 con 0x7f451c109a00 2024-05-03T18:37:35.649 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:34.980+0000 7f4512ffd700 1 -- 172.21.15.32:0/4279498952 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7f451800b3e0 con 0x7f451c109a00 2024-05-03T18:37:35.649 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:34.983+0000 7f4523bca700 1 -- 172.21.15.32:0/4279498952 >> [v2:172.21.15.32:6800/2019702098,v1:172.21.15.32:6801/2019702098] conn(0x7f450803e4d0 msgr2=0x7f4508040990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:37:35.649 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:34.983+0000 7f4523bca700 1 --2- 172.21.15.32:0/4279498952 >> [v2:172.21.15.32:6800/2019702098,v1:172.21.15.32:6801/2019702098] conn(0x7f450803e4d0 0x7f4508040990 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f450c006fd0 tx=0x7f450c006e40 comp rx=0 tx=0).stop 2024-05-03T18:37:35.649 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:34.983+0000 7f4523bca700 1 -- 172.21.15.32:0/4279498952 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f451c109a00 msgr2=0x7f451c19f0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:37:35.649 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:34.983+0000 7f4523bca700 1 --2- 172.21.15.32:0/4279498952 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f451c109a00 0x7f451c19f0f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f45180116e0 tx=0x7f4518011aa0 comp rx=0 tx=0).stop 2024-05-03T18:37:35.649 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:34.983+0000 7f4523bca700 1 -- 172.21.15.32:0/4279498952 shutdown_connections 2024-05-03T18:37:35.649 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:34.983+0000 7f4523bca700 1 --2- 172.21.15.32:0/4279498952 >> [v2:172.21.15.32:6800/2019702098,v1:172.21.15.32:6801/2019702098] conn(0x7f450803e4d0 0x7f4508040990 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:35.649 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:34.983+0000 7f4523bca700 1 --2- 172.21.15.32:0/4279498952 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f451c109a00 0x7f451c19f0f0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:35.649 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:34.983+0000 7f4523bca700 1 -- 172.21.15.32:0/4279498952 >> 172.21.15.32:0/4279498952 conn(0x7f451c1034d0 msgr2=0x7f451c103fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:37:35.649 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:34.983+0000 7f4523bca700 1 -- 172.21.15.32:0/4279498952 shutdown_connections 2024-05-03T18:37:35.649 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:34.983+0000 7f4523bca700 1 -- 172.21.15.32:0/4279498952 wait complete. 2024-05-03T18:37:36.108 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:37:35 smithi032 ceph-mon[29763]: from='client.? 172.21.15.32:0/4279498952' entity='client.admin' 2024-05-03T18:37:36.913 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:36.147+0000 7f1febbfa700 1 Processor -- start 2024-05-03T18:37:36.913 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:36.147+0000 7f1febbfa700 1 -- start start 2024-05-03T18:37:36.913 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:36.147+0000 7f1febbfa700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1fe4109010 0x7f1fe4109430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:36.913 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:36.147+0000 7f1febbfa700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe4109a00 con 0x7f1fe4109010 2024-05-03T18:37:36.913 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:36.147+0000 7f1fe9996700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1fe4109010 0x7f1fe4109430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:36.913 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:36.147+0000 7f1fe9996700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1fe4109010 0x7f1fe4109430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38716/0 (socket says 172.21.15.32:38716) 2024-05-03T18:37:36.914 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:36.147+0000 7f1fe9996700 1 -- 172.21.15.32:0/2010377337 learned_addr learned my addr 172.21.15.32:0/2010377337 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:37:36.914 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:36.148+0000 7f1fe9996700 1 -- 172.21.15.32:0/2010377337 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fe410a170 con 0x7f1fe4109010 2024-05-03T18:37:36.914 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:36.148+0000 7f1fe9996700 1 --2- 172.21.15.32:0/2010377337 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1fe4109010 0x7f1fe4109430 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1fe00098a0 tx=0x7f1fe0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d5969a8dcebbe43 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:37:36.914 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:36.148+0000 7f1fe8994700 1 -- 172.21.15.32:0/2010377337 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1fe0004160 con 0x7f1fe4109010 2024-05-03T18:37:36.914 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:36.148+0000 7f1fe8994700 1 -- 172.21.15.32:0/2010377337 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1fe0004300 con 0x7f1fe4109010 2024-05-03T18:37:36.914 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:36.148+0000 7f1fe8994700 1 -- 172.21.15.32:0/2010377337 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1fe0011a20 con 0x7f1fe4109010 2024-05-03T18:37:36.914 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:36.149+0000 7f1febbfa700 1 -- 172.21.15.32:0/2010377337 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1fe4109010 msgr2=0x7f1fe4109430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:37:36.914 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:36.149+0000 7f1febbfa700 1 --2- 172.21.15.32:0/2010377337 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1fe4109010 0x7f1fe4109430 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1fe00098a0 tx=0x7f1fe0009bb0 comp rx=0 tx=0).stop 2024-05-03T18:37:36.914 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:36.149+0000 7f1febbfa700 1 -- 172.21.15.32:0/2010377337 shutdown_connections 2024-05-03T18:37:36.914 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:36.149+0000 7f1febbfa700 1 --2- 172.21.15.32:0/2010377337 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1fe4109010 0x7f1fe4109430 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:36.914 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:36.149+0000 7f1febbfa700 1 -- 172.21.15.32:0/2010377337 >> 172.21.15.32:0/2010377337 conn(0x7f1fe4104650 msgr2=0x7f1fe4106a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:37:36.914 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:36.149+0000 7f1febbfa700 1 -- 172.21.15.32:0/2010377337 shutdown_connections 2024-05-03T18:37:36.914 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:36.149+0000 7f1febbfa700 1 -- 172.21.15.32:0/2010377337 wait complete. 2024-05-03T18:37:36.915 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:36.150+0000 7f1febbfa700 1 Processor -- start 2024-05-03T18:37:36.915 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:36.150+0000 7f1febbfa700 1 -- start start 2024-05-03T18:37:36.915 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:36.150+0000 7f1febbfa700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1fe4109010 0x7f1fe407eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:36.915 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:36.150+0000 7f1febbfa700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe410d520 con 0x7f1fe4109010 2024-05-03T18:37:36.915 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:36.150+0000 7f1fe9996700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1fe4109010 0x7f1fe407eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:36.915 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:36.150+0000 7f1fe9996700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1fe4109010 0x7f1fe407eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38728/0 (socket says 172.21.15.32:38728) 2024-05-03T18:37:36.915 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:36.150+0000 7f1fe9996700 1 -- 172.21.15.32:0/264995327 learned_addr learned my addr 172.21.15.32:0/264995327 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:37:36.915 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:36.150+0000 7f1fe9996700 1 -- 172.21.15.32:0/264995327 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fe407f0c0 con 0x7f1fe4109010 2024-05-03T18:37:36.915 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:36.150+0000 7f1fe9996700 1 --2- 172.21.15.32:0/264995327 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1fe4109010 0x7f1fe407eb80 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1fe00119f0 tx=0x7f1fe0011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:37:36.915 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:36.151+0000 7f1fdaffd700 1 -- 172.21.15.32:0/264995327 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1fe00039e0 con 0x7f1fe4109010 2024-05-03T18:37:36.915 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:36.151+0000 7f1fdaffd700 1 -- 172.21.15.32:0/264995327 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1fe001b8f0 con 0x7f1fe4109010 2024-05-03T18:37:36.915 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:36.151+0000 7f1febbfa700 1 -- 172.21.15.32:0/264995327 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fe407d2b0 con 0x7f1fe4109010 2024-05-03T18:37:36.915 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:36.151+0000 7f1fdaffd700 1 -- 172.21.15.32:0/264995327 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1fe00237a0 con 0x7f1fe4109010 2024-05-03T18:37:36.915 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:36.151+0000 7f1febbfa700 1 -- 172.21.15.32:0/264995327 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fe407d7f0 con 0x7f1fe4109010 2024-05-03T18:37:36.916 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:36.152+0000 7f1febbfa700 1 -- 172.21.15.32:0/264995327 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1fe4064b00 con 0x7f1fe4109010 2024-05-03T18:37:36.916 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:36.152+0000 7f1fdaffd700 1 -- 172.21.15.32:0/264995327 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f1fe002b070 con 0x7f1fe4109010 2024-05-03T18:37:36.916 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:36.152+0000 7f1fdaffd700 1 --2- 172.21.15.32:0/264995327 >> [v2:172.21.15.32:6800/2019702098,v1:172.21.15.32:6801/2019702098] conn(0x7f1fd003e420 0x7f1fd00408e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:36.916 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:36.152+0000 7f1fdaffd700 1 -- 172.21.15.32:0/264995327 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f1fe0053e40 con 0x7f1fe4109010 2024-05-03T18:37:36.916 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:36.152+0000 7f1fe9195700 1 --2- 172.21.15.32:0/264995327 >> [v2:172.21.15.32:6800/2019702098,v1:172.21.15.32:6801/2019702098] conn(0x7f1fd003e420 0x7f1fd00408e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:36.916 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:36.153+0000 7f1fe9195700 1 --2- 172.21.15.32:0/264995327 >> [v2:172.21.15.32:6800/2019702098,v1:172.21.15.32:6801/2019702098] conn(0x7f1fd003e420 0x7f1fd00408e0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f1fd4006fd0 tx=0x7f1fd4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:37:36.916 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:36.157+0000 7f1fdaffd700 1 -- 172.21.15.32:0/264995327 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f1fe001f030 con 0x7f1fe4109010 2024-05-03T18:37:36.916 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:36.372+0000 7f1febbfa700 1 -- 172.21.15.32:0/264995327 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7f1fe407da70 con 0x7f1fe4109010 2024-05-03T18:37:36.916 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:36.375+0000 7f1fdaffd700 1 -- 172.21.15.32:0/264995327 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7f1fe001baa0 con 0x7f1fe4109010 2024-05-03T18:37:36.916 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:36.377+0000 7f1febbfa700 1 -- 172.21.15.32:0/264995327 >> [v2:172.21.15.32:6800/2019702098,v1:172.21.15.32:6801/2019702098] conn(0x7f1fd003e420 msgr2=0x7f1fd00408e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:37:36.916 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:36.377+0000 7f1febbfa700 1 --2- 172.21.15.32:0/264995327 >> [v2:172.21.15.32:6800/2019702098,v1:172.21.15.32:6801/2019702098] conn(0x7f1fd003e420 0x7f1fd00408e0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f1fd4006fd0 tx=0x7f1fd4006e40 comp rx=0 tx=0).stop 2024-05-03T18:37:36.916 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:36.377+0000 7f1febbfa700 1 -- 172.21.15.32:0/264995327 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1fe4109010 msgr2=0x7f1fe407eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:37:36.916 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:36.377+0000 7f1febbfa700 1 --2- 172.21.15.32:0/264995327 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1fe4109010 0x7f1fe407eb80 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1fe00119f0 tx=0x7f1fe0011ad0 comp rx=0 tx=0).stop 2024-05-03T18:37:36.916 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:36.378+0000 7f1febbfa700 1 -- 172.21.15.32:0/264995327 shutdown_connections 2024-05-03T18:37:36.916 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:36.378+0000 7f1febbfa700 1 --2- 172.21.15.32:0/264995327 >> [v2:172.21.15.32:6800/2019702098,v1:172.21.15.32:6801/2019702098] conn(0x7f1fd003e420 0x7f1fd00408e0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:36.916 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:36.378+0000 7f1febbfa700 1 --2- 172.21.15.32:0/264995327 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1fe4109010 0x7f1fe407eb80 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:36.917 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:36.378+0000 7f1febbfa700 1 -- 172.21.15.32:0/264995327 >> 172.21.15.32:0/264995327 conn(0x7f1fe4104650 msgr2=0x7f1fe4105170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:37:36.917 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:36.378+0000 7f1febbfa700 1 -- 172.21.15.32:0/264995327 shutdown_connections 2024-05-03T18:37:36.917 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:36.378+0000 7f1febbfa700 1 -- 172.21.15.32:0/264995327 wait complete. 2024-05-03T18:37:36.917 INFO:teuthology.orchestra.run.smithi032.stdout:Enabling the dashboard module... 2024-05-03T18:37:37.567 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:37:37 smithi032 ceph-mon[29763]: from='client.? 172.21.15.32:0/264995327' entity='client.admin' 2024-05-03T18:37:38.737 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:37:38 smithi032 ceph-mon[29763]: from='mgr.14120 172.21.15.32:0/2820864439' entity='mgr.x' 2024-05-03T18:37:38.737 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:37:38 smithi032 ceph-mon[29763]: from='client.? 172.21.15.32:0/2898064629' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-05-03T18:37:38.737 INFO:journalctl@ceph.mgr.x.smithi032.stdout:May 03 18:37:38 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-x[29995]: ignoring --setuser ceph since I am not root 2024-05-03T18:37:38.737 INFO:journalctl@ceph.mgr.x.smithi032.stdout:May 03 18:37:38 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-x[29995]: ignoring --setgroup ceph since I am not root 2024-05-03T18:37:38.737 INFO:journalctl@ceph.mgr.x.smithi032.stdout:May 03 18:37:38 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-x[29995]: 2024-05-03T18:37:38.622+0000 7fed36d7a200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-03T18:37:39.021 INFO:journalctl@ceph.mgr.x.smithi032.stdout:May 03 18:37:38 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-x[29995]: 2024-05-03T18:37:38.816+0000 7fed36d7a200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-03T18:37:39.022 INFO:journalctl@ceph.mgr.x.smithi032.stdout:May 03 18:37:38 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-x[29995]: 2024-05-03T18:37:38.922+0000 7fed36d7a200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-03T18:37:39.035 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:37.338+0000 7f3144ad0700 1 Processor -- start 2024-05-03T18:37:39.035 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:37.338+0000 7f3144ad0700 1 -- start start 2024-05-03T18:37:39.035 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:37.339+0000 7f3144ad0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3140109ac0 0x7f314010beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:39.035 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:37.339+0000 7f3144ad0700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f3140076ce0 con 0x7f3140109ac0 2024-05-03T18:37:39.035 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:37.339+0000 7f313e59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3140109ac0 0x7f314010beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:39.035 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:37.339+0000 7f313e59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3140109ac0 0x7f314010beb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38744/0 (socket says 172.21.15.32:38744) 2024-05-03T18:37:39.036 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:37.339+0000 7f313e59c700 1 -- 172.21.15.32:0/2994489585 learned_addr learned my addr 172.21.15.32:0/2994489585 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:37:39.036 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:37.339+0000 7f313e59c700 1 -- 172.21.15.32:0/2994489585 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f314010c3f0 con 0x7f3140109ac0 2024-05-03T18:37:39.036 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:37.339+0000 7f313e59c700 1 --2- 172.21.15.32:0/2994489585 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3140109ac0 0x7f314010beb0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f31300098a0 tx=0x7f3130009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c68e14f619d2e46 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:37:39.036 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:37.339+0000 7f313d59a700 1 -- 172.21.15.32:0/2994489585 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3130004160 con 0x7f3140109ac0 2024-05-03T18:37:39.036 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:37.340+0000 7f313d59a700 1 -- 172.21.15.32:0/2994489585 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3130004300 con 0x7f3140109ac0 2024-05-03T18:37:39.036 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:37.340+0000 7f313d59a700 1 -- 172.21.15.32:0/2994489585 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3130011a20 con 0x7f3140109ac0 2024-05-03T18:37:39.037 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:37.340+0000 7f3144ad0700 1 -- 172.21.15.32:0/2994489585 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3140109ac0 msgr2=0x7f314010beb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:37:39.037 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:37.340+0000 7f3144ad0700 1 --2- 172.21.15.32:0/2994489585 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3140109ac0 0x7f314010beb0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f31300098a0 tx=0x7f3130009bb0 comp rx=0 tx=0).stop 2024-05-03T18:37:39.037 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:37.340+0000 7f3144ad0700 1 -- 172.21.15.32:0/2994489585 shutdown_connections 2024-05-03T18:37:39.037 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:37.340+0000 7f3144ad0700 1 --2- 172.21.15.32:0/2994489585 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3140109ac0 0x7f314010beb0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:39.037 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:37.340+0000 7f3144ad0700 1 -- 172.21.15.32:0/2994489585 >> 172.21.15.32:0/2994489585 conn(0x7f31401034d0 msgr2=0x7f31401058f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:37:39.037 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:37.340+0000 7f3144ad0700 1 -- 172.21.15.32:0/2994489585 shutdown_connections 2024-05-03T18:37:39.038 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:37.340+0000 7f3144ad0700 1 -- 172.21.15.32:0/2994489585 wait complete. 2024-05-03T18:37:39.038 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:37.341+0000 7f3144ad0700 1 Processor -- start 2024-05-03T18:37:39.038 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:37.341+0000 7f3144ad0700 1 -- start start 2024-05-03T18:37:39.038 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:37.341+0000 7f3144ad0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3140109ac0 0x7f314019ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:39.038 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:37.341+0000 7f3144ad0700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f314010d340 con 0x7f3140109ac0 2024-05-03T18:37:39.038 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:37.341+0000 7f313e59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3140109ac0 0x7f314019ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:39.039 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:37.341+0000 7f313e59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3140109ac0 0x7f314019ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38760/0 (socket says 172.21.15.32:38760) 2024-05-03T18:37:39.039 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:37.342+0000 7f313e59c700 1 -- 172.21.15.32:0/2898064629 learned_addr learned my addr 172.21.15.32:0/2898064629 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:37:39.039 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:37.342+0000 7f313e59c700 1 -- 172.21.15.32:0/2898064629 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f314019d380 con 0x7f3140109ac0 2024-05-03T18:37:39.039 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:37.342+0000 7f313e59c700 1 --2- 172.21.15.32:0/2898064629 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3140109ac0 0x7f314019ce40 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3130000c00 tx=0x7f3130011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:37:39.039 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:37.342+0000 7f312f7fe700 1 -- 172.21.15.32:0/2898064629 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3130003aa0 con 0x7f3140109ac0 2024-05-03T18:37:39.039 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:37.342+0000 7f312f7fe700 1 -- 172.21.15.32:0/2898064629 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f313001b8f0 con 0x7f3140109ac0 2024-05-03T18:37:39.040 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:37.342+0000 7f3144ad0700 1 -- 172.21.15.32:0/2898064629 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f314019d610 con 0x7f3140109ac0 2024-05-03T18:37:39.040 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:37.342+0000 7f312f7fe700 1 -- 172.21.15.32:0/2898064629 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3130023870 con 0x7f3140109ac0 2024-05-03T18:37:39.040 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:37.342+0000 7f3144ad0700 1 -- 172.21.15.32:0/2898064629 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31401a0300 con 0x7f3140109ac0 2024-05-03T18:37:39.040 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:37.343+0000 7f3144ad0700 1 -- 172.21.15.32:0/2898064629 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3140064b00 con 0x7f3140109ac0 2024-05-03T18:37:39.040 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:37.343+0000 7f312f7fe700 1 -- 172.21.15.32:0/2898064629 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f3130023b20 con 0x7f3140109ac0 2024-05-03T18:37:39.040 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:37.343+0000 7f312f7fe700 1 --2- 172.21.15.32:0/2898064629 >> [v2:172.21.15.32:6800/2019702098,v1:172.21.15.32:6801/2019702098] conn(0x7f312803e4c0 0x7f3128040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:39.040 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:37.344+0000 7f312f7fe700 1 -- 172.21.15.32:0/2898064629 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f3130053ed0 con 0x7f3140109ac0 2024-05-03T18:37:39.040 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:37.344+0000 7f313dd9b700 1 --2- 172.21.15.32:0/2898064629 >> [v2:172.21.15.32:6800/2019702098,v1:172.21.15.32:6801/2019702098] conn(0x7f312803e4c0 0x7f3128040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:39.040 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:37.344+0000 7f313dd9b700 1 --2- 172.21.15.32:0/2898064629 >> [v2:172.21.15.32:6800/2019702098,v1:172.21.15.32:6801/2019702098] conn(0x7f312803e4c0 0x7f3128040980 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f3134006fd0 tx=0x7f3134006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:37:39.040 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:37.348+0000 7f312f7fe700 1 -- 172.21.15.32:0/2898064629 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f313001f030 con 0x7f3140109ac0 2024-05-03T18:37:39.041 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:37.632+0000 7f3144ad0700 1 -- 172.21.15.32:0/2898064629 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7f31401a08f0 con 0x7f3140109ac0 2024-05-03T18:37:39.041 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:38.421+0000 7f312f7fe700 1 -- 172.21.15.32:0/2898064629 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7f313001baa0 con 0x7f3140109ac0 2024-05-03T18:37:39.041 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:38.423+0000 7f312f7fe700 1 -- 172.21.15.32:0/2898064629 <== mon.0 v2:172.21.15.32:3300/0 8 ==== mgrmap(e 9) ==== 50235+0+0 (secure 0 0 0) 0x7f313002ba40 con 0x7f3140109ac0 2024-05-03T18:37:39.041 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:38.424+0000 7f3144ad0700 1 -- 172.21.15.32:0/2898064629 >> [v2:172.21.15.32:6800/2019702098,v1:172.21.15.32:6801/2019702098] conn(0x7f312803e4c0 msgr2=0x7f3128040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:37:39.041 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:38.424+0000 7f3144ad0700 1 --2- 172.21.15.32:0/2898064629 >> [v2:172.21.15.32:6800/2019702098,v1:172.21.15.32:6801/2019702098] conn(0x7f312803e4c0 0x7f3128040980 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f3134006fd0 tx=0x7f3134006e40 comp rx=0 tx=0).stop 2024-05-03T18:37:39.041 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:38.424+0000 7f3144ad0700 1 -- 172.21.15.32:0/2898064629 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3140109ac0 msgr2=0x7f314019ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:37:39.041 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:38.424+0000 7f3144ad0700 1 --2- 172.21.15.32:0/2898064629 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3140109ac0 0x7f314019ce40 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3130000c00 tx=0x7f3130011aa0 comp rx=0 tx=0).stop 2024-05-03T18:37:39.041 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:38.424+0000 7f3144ad0700 1 -- 172.21.15.32:0/2898064629 shutdown_connections 2024-05-03T18:37:39.041 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:38.424+0000 7f3144ad0700 1 --2- 172.21.15.32:0/2898064629 >> [v2:172.21.15.32:6800/2019702098,v1:172.21.15.32:6801/2019702098] conn(0x7f312803e4c0 0x7f3128040980 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:39.041 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:38.424+0000 7f3144ad0700 1 --2- 172.21.15.32:0/2898064629 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3140109ac0 0x7f314019ce40 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:39.041 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:38.424+0000 7f3144ad0700 1 -- 172.21.15.32:0/2898064629 >> 172.21.15.32:0/2898064629 conn(0x7f31401034d0 msgr2=0x7f3140103fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:37:39.041 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:38.424+0000 7f3144ad0700 1 -- 172.21.15.32:0/2898064629 shutdown_connections 2024-05-03T18:37:39.041 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:38.424+0000 7f3144ad0700 1 -- 172.21.15.32:0/2898064629 wait complete. 2024-05-03T18:37:39.317 INFO:journalctl@ceph.mgr.x.smithi032.stdout:May 03 18:37:39 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-x[29995]: 2024-05-03T18:37:39.257+0000 7fed36d7a200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-03T18:37:39.677 INFO:journalctl@ceph.mgr.x.smithi032.stdout:May 03 18:37:39 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-x[29995]: 2024-05-03T18:37:39.346+0000 7fed36d7a200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-03T18:37:39.677 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:37:39 smithi032 ceph-mon[29763]: from='client.? 172.21.15.32:0/2898064629' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-05-03T18:37:39.677 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:37:39 smithi032 ceph-mon[29763]: mgrmap e9: x(active, since 19s) 2024-05-03T18:37:40.310 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout { 2024-05-03T18:37:40.310 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-05-03T18:37:40.310 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-03T18:37:40.310 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-05-03T18:37:40.310 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-05-03T18:37:40.310 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout } 2024-05-03T18:37:40.310 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:39.469+0000 7f58dc691700 1 Processor -- start 2024-05-03T18:37:40.310 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:39.469+0000 7f58dc691700 1 -- start start 2024-05-03T18:37:40.310 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:39.469+0000 7f58dc691700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58d4109a30 0x7f58d410be20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:40.311 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:39.469+0000 7f58dc691700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f58d407cec0 con 0x7f58d4109a30 2024-05-03T18:37:40.311 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:39.470+0000 7f58da42d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58d4109a30 0x7f58d410be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:40.311 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:39.470+0000 7f58da42d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58d4109a30 0x7f58d410be20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38794/0 (socket says 172.21.15.32:38794) 2024-05-03T18:37:40.311 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:39.470+0000 7f58da42d700 1 -- 172.21.15.32:0/3181674717 learned_addr learned my addr 172.21.15.32:0/3181674717 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:37:40.311 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:39.470+0000 7f58da42d700 1 -- 172.21.15.32:0/3181674717 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58d410c360 con 0x7f58d4109a30 2024-05-03T18:37:40.311 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:39.470+0000 7f58da42d700 1 --2- 172.21.15.32:0/3181674717 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58d4109a30 0x7f58d410be20 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f58c40098a0 tx=0x7f58c4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=daef5e114657969f server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:37:40.311 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:39.470+0000 7f58d942b700 1 -- 172.21.15.32:0/3181674717 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f58c4004160 con 0x7f58d4109a30 2024-05-03T18:37:40.311 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:39.470+0000 7f58d942b700 1 -- 172.21.15.32:0/3181674717 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f58c4004300 con 0x7f58d4109a30 2024-05-03T18:37:40.311 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:39.470+0000 7f58d942b700 1 -- 172.21.15.32:0/3181674717 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f58c4011a20 con 0x7f58d4109a30 2024-05-03T18:37:40.311 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:39.471+0000 7f58dc691700 1 -- 172.21.15.32:0/3181674717 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58d4109a30 msgr2=0x7f58d410be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:37:40.311 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:39.471+0000 7f58dc691700 1 --2- 172.21.15.32:0/3181674717 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58d4109a30 0x7f58d410be20 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f58c40098a0 tx=0x7f58c4009bb0 comp rx=0 tx=0).stop 2024-05-03T18:37:40.312 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:39.471+0000 7f58dc691700 1 -- 172.21.15.32:0/3181674717 shutdown_connections 2024-05-03T18:37:40.312 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:39.471+0000 7f58dc691700 1 --2- 172.21.15.32:0/3181674717 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58d4109a30 0x7f58d410be20 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:40.312 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:39.471+0000 7f58dc691700 1 -- 172.21.15.32:0/3181674717 >> 172.21.15.32:0/3181674717 conn(0x7f58d41034c0 msgr2=0x7f58d41058e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:37:40.312 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:39.471+0000 7f58dc691700 1 -- 172.21.15.32:0/3181674717 shutdown_connections 2024-05-03T18:37:40.312 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:39.471+0000 7f58dc691700 1 -- 172.21.15.32:0/3181674717 wait complete. 2024-05-03T18:37:40.312 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:39.472+0000 7f58dc691700 1 Processor -- start 2024-05-03T18:37:40.312 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:39.472+0000 7f58dc691700 1 -- start start 2024-05-03T18:37:40.312 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:39.472+0000 7f58dc691700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58d4109a30 0x7f58d419ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:40.312 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:39.472+0000 7f58dc691700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f58d410d370 con 0x7f58d4109a30 2024-05-03T18:37:40.312 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:39.472+0000 7f58da42d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58d4109a30 0x7f58d419ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:40.312 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:39.472+0000 7f58da42d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58d4109a30 0x7f58d419ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38808/0 (socket says 172.21.15.32:38808) 2024-05-03T18:37:40.312 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:39.472+0000 7f58da42d700 1 -- 172.21.15.32:0/705935346 learned_addr learned my addr 172.21.15.32:0/705935346 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:37:40.312 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:39.473+0000 7f58da42d700 1 -- 172.21.15.32:0/705935346 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58d419d3b0 con 0x7f58d4109a30 2024-05-03T18:37:40.313 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:39.473+0000 7f58da42d700 1 --2- 172.21.15.32:0/705935346 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58d4109a30 0x7f58d419ce70 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f58c4000c00 tx=0x7f58c4011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:37:40.313 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:39.473+0000 7f58cb7fe700 1 -- 172.21.15.32:0/705935346 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f58c4003aa0 con 0x7f58d4109a30 2024-05-03T18:37:40.313 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:39.473+0000 7f58cb7fe700 1 -- 172.21.15.32:0/705935346 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f58c401b8f0 con 0x7f58d4109a30 2024-05-03T18:37:40.313 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:39.473+0000 7f58dc691700 1 -- 172.21.15.32:0/705935346 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58d419d640 con 0x7f58d4109a30 2024-05-03T18:37:40.313 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:39.473+0000 7f58cb7fe700 1 -- 172.21.15.32:0/705935346 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f58c4023870 con 0x7f58d4109a30 2024-05-03T18:37:40.313 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:39.473+0000 7f58dc691700 1 -- 172.21.15.32:0/705935346 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58d41a0330 con 0x7f58d4109a30 2024-05-03T18:37:40.313 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:39.474+0000 7f58cb7fe700 1 -- 172.21.15.32:0/705935346 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 9) ==== 50235+0+0 (secure 0 0 0) 0x7f58c4023b70 con 0x7f58d4109a30 2024-05-03T18:37:40.313 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:39.474+0000 7f58dc691700 1 -- 172.21.15.32:0/705935346 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58d4064b00 con 0x7f58d4109a30 2024-05-03T18:37:40.313 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:39.474+0000 7f58cb7fe700 1 --2- 172.21.15.32:0/705935346 >> [v2:172.21.15.32:6800/2019702098,v1:172.21.15.32:6801/2019702098] conn(0x7f58c003e4c0 0x7f58c0040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:40.313 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:39.475+0000 7f58cb7fe700 1 -- 172.21.15.32:0/705935346 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f58c4053fc0 con 0x7f58d4109a30 2024-05-03T18:37:40.313 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:39.475+0000 7f58d9c2c700 1 -- 172.21.15.32:0/705935346 >> [v2:172.21.15.32:6800/2019702098,v1:172.21.15.32:6801/2019702098] conn(0x7f58c003e4c0 msgr2=0x7f58c0040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6800/2019702098 2024-05-03T18:37:40.313 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:39.475+0000 7f58d9c2c700 1 --2- 172.21.15.32:0/705935346 >> [v2:172.21.15.32:6800/2019702098,v1:172.21.15.32:6801/2019702098] conn(0x7f58c003e4c0 0x7f58c0040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-03T18:37:40.313 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:39.480+0000 7f58cb7fe700 1 -- 172.21.15.32:0/705935346 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f58c401f030 con 0x7f58d4109a30 2024-05-03T18:37:40.313 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:39.676+0000 7f58d9c2c700 1 -- 172.21.15.32:0/705935346 >> [v2:172.21.15.32:6800/2019702098,v1:172.21.15.32:6801/2019702098] conn(0x7f58c003e4c0 msgr2=0x7f58c0040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6800/2019702098 2024-05-03T18:37:40.314 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:39.676+0000 7f58d9c2c700 1 --2- 172.21.15.32:0/705935346 >> [v2:172.21.15.32:6800/2019702098,v1:172.21.15.32:6801/2019702098] conn(0x7f58c003e4c0 0x7f58c0040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-03T18:37:40.314 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:39.774+0000 7f58dc691700 1 -- 172.21.15.32:0/705935346 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f58d41a0920 con 0x7f58d4109a30 2024-05-03T18:37:40.314 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:39.775+0000 7f58cb7fe700 1 -- 172.21.15.32:0/705935346 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7f58c400be20 con 0x7f58d4109a30 2024-05-03T18:37:40.314 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:39.778+0000 7f58dc691700 1 -- 172.21.15.32:0/705935346 >> [v2:172.21.15.32:6800/2019702098,v1:172.21.15.32:6801/2019702098] conn(0x7f58c003e4c0 msgr2=0x7f58c0040980 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-03T18:37:40.314 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:39.779+0000 7f58dc691700 1 --2- 172.21.15.32:0/705935346 >> [v2:172.21.15.32:6800/2019702098,v1:172.21.15.32:6801/2019702098] conn(0x7f58c003e4c0 0x7f58c0040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:40.314 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:39.779+0000 7f58dc691700 1 -- 172.21.15.32:0/705935346 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58d4109a30 msgr2=0x7f58d419ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:37:40.314 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:39.779+0000 7f58dc691700 1 --2- 172.21.15.32:0/705935346 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58d4109a30 0x7f58d419ce70 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f58c4000c00 tx=0x7f58c4011aa0 comp rx=0 tx=0).stop 2024-05-03T18:37:40.314 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:39.779+0000 7f58dc691700 1 -- 172.21.15.32:0/705935346 shutdown_connections 2024-05-03T18:37:40.314 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:39.779+0000 7f58dc691700 1 --2- 172.21.15.32:0/705935346 >> [v2:172.21.15.32:6800/2019702098,v1:172.21.15.32:6801/2019702098] conn(0x7f58c003e4c0 0x7f58c0040980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:40.314 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:39.779+0000 7f58dc691700 1 --2- 172.21.15.32:0/705935346 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58d4109a30 0x7f58d419ce70 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:40.314 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:39.779+0000 7f58dc691700 1 -- 172.21.15.32:0/705935346 >> 172.21.15.32:0/705935346 conn(0x7f58d41034c0 msgr2=0x7f58d4103f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:37:40.314 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:39.781+0000 7f58dc691700 1 -- 172.21.15.32:0/705935346 shutdown_connections 2024-05-03T18:37:40.315 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:39.781+0000 7f58dc691700 1 -- 172.21.15.32:0/705935346 wait complete. 2024-05-03T18:37:40.315 INFO:teuthology.orchestra.run.smithi032.stdout:Waiting for the mgr to restart... 2024-05-03T18:37:40.315 INFO:teuthology.orchestra.run.smithi032.stdout:Waiting for mgr epoch 9... 2024-05-03T18:37:40.722 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:37:40 smithi032 ceph-mon[29763]: from='client.? 172.21.15.32:0/705935346' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-05-03T18:37:40.722 INFO:journalctl@ceph.mgr.x.smithi032.stdout:May 03 18:37:40 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-x[29995]: 2024-05-03T18:37:40.622+0000 7fed36d7a200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-03T18:37:40.984 INFO:journalctl@ceph.mgr.x.smithi032.stdout:May 03 18:37:40 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-x[29995]: 2024-05-03T18:37:40.806+0000 7fed36d7a200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-03T18:37:40.984 INFO:journalctl@ceph.mgr.x.smithi032.stdout:May 03 18:37:40 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-x[29995]: 2024-05-03T18:37:40.983+0000 7fed36d7a200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-03T18:37:41.568 INFO:journalctl@ceph.mgr.x.smithi032.stdout:May 03 18:37:41 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-x[29995]: 2024-05-03T18:37:41.285+0000 7fed36d7a200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-03T18:37:41.568 INFO:journalctl@ceph.mgr.x.smithi032.stdout:May 03 18:37:41 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-x[29995]: 2024-05-03T18:37:41.382+0000 7fed36d7a200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-03T18:37:42.067 INFO:journalctl@ceph.mgr.x.smithi032.stdout:May 03 18:37:41 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-x[29995]: 2024-05-03T18:37:41.593+0000 7fed36d7a200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-03T18:37:42.068 INFO:journalctl@ceph.mgr.x.smithi032.stdout:May 03 18:37:41 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-x[29995]: 2024-05-03T18:37:41.793+0000 7fed36d7a200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-03T18:37:42.818 INFO:journalctl@ceph.mgr.x.smithi032.stdout:May 03 18:37:42 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-x[29995]: 2024-05-03T18:37:42.425+0000 7fed36d7a200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-03T18:37:42.818 INFO:journalctl@ceph.mgr.x.smithi032.stdout:May 03 18:37:42 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-x[29995]: 2024-05-03T18:37:42.657+0000 7fed36d7a200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-03T18:37:43.567 INFO:journalctl@ceph.mgr.x.smithi032.stdout:May 03 18:37:43 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-x[29995]: 2024-05-03T18:37:43.099+0000 7fed36d7a200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-03T18:37:43.568 INFO:journalctl@ceph.mgr.x.smithi032.stdout:May 03 18:37:43 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-x[29995]: 2024-05-03T18:37:43.200+0000 7fed36d7a200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-03T18:37:43.987 INFO:journalctl@ceph.mgr.x.smithi032.stdout:May 03 18:37:43 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-x[29995]: 2024-05-03T18:37:43.610+0000 7fed36d7a200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-03T18:37:44.317 INFO:journalctl@ceph.mgr.x.smithi032.stdout:May 03 18:37:43 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-x[29995]: 2024-05-03T18:37:43.986+0000 7fed36d7a200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-03T18:37:44.318 INFO:journalctl@ceph.mgr.x.smithi032.stdout:May 03 18:37:44 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-x[29995]: 2024-05-03T18:37:44.083+0000 7fed36d7a200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-03T18:37:44.796 INFO:journalctl@ceph.mgr.x.smithi032.stdout:May 03 18:37:44 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-x[29995]: 2024-05-03T18:37:44.340+0000 7fed36d7a200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-03T18:37:44.796 INFO:journalctl@ceph.mgr.x.smithi032.stdout:May 03 18:37:44 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-x[29995]: 2024-05-03T18:37:44.480+0000 7fed36d7a200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-03T18:37:44.796 INFO:journalctl@ceph.mgr.x.smithi032.stdout:May 03 18:37:44 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-x[29995]: 2024-05-03T18:37:44.795+0000 7fed36d7a200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-03T18:37:45.068 INFO:journalctl@ceph.mgr.x.smithi032.stdout:May 03 18:37:44 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-x[29995]: 2024-05-03T18:37:44.916+0000 7fed36d7a200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-03T18:37:45.068 INFO:journalctl@ceph.mgr.x.smithi032.stdout:May 03 18:37:45 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-x[29995]: 2024-05-03T18:37:45.011+0000 7fed36d7a200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-03T18:37:45.931 INFO:journalctl@ceph.mgr.x.smithi032.stdout:May 03 18:37:45 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-x[29995]: 2024-05-03T18:37:45.646+0000 7fed36d7a200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-03T18:37:45.931 INFO:journalctl@ceph.mgr.x.smithi032.stdout:May 03 18:37:45 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-x[29995]: 2024-05-03T18:37:45.877+0000 7fed36d7a200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-03T18:37:46.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:37:45 smithi032 ceph-mon[29763]: Active manager daemon x restarted 2024-05-03T18:37:46.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:37:45 smithi032 ceph-mon[29763]: Activating manager daemon x 2024-05-03T18:37:46.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:37:45 smithi032 ceph-mon[29763]: osdmap e3: 0 total, 0 up, 0 in 2024-05-03T18:37:46.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:37:45 smithi032 ceph-mon[29763]: mgrmap e10: x(active, starting, since 0.00363634s) 2024-05-03T18:37:46.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:37:45 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-03T18:37:46.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:37:45 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-03T18:37:46.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:37:45 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-03T18:37:46.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:37:45 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-03T18:37:46.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:37:45 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-03T18:37:46.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:37:45 smithi032 ceph-mon[29763]: Manager daemon x is now available 2024-05-03T18:37:46.319 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:37:45 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:37:47.081 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:37:46 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-03T18:37:47.082 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:37:46 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-03T18:37:47.082 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:37:46 smithi032 ceph-mon[29763]: [03/May/2024:18:37:46] ENGINE Bus STARTING 2024-05-03T18:37:47.082 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:37:46 smithi032 ceph-mon[29763]: [03/May/2024:18:37:46] ENGINE Serving on http://172.21.15.32:8765 2024-05-03T18:37:47.082 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:37:46 smithi032 ceph-mon[29763]: mgrmap e11: x(active, since 1.00614s) 2024-05-03T18:37:47.723 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout { 2024-05-03T18:37:47.723 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-05-03T18:37:47.723 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout "initialized": true 2024-05-03T18:37:47.723 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout } 2024-05-03T18:37:47.723 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:40.750+0000 7fbf970e4700 1 Processor -- start 2024-05-03T18:37:47.723 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:40.750+0000 7fbf970e4700 1 -- start start 2024-05-03T18:37:47.723 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:40.750+0000 7fbf970e4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf90107890 0x7fbf90107cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:47.724 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:40.750+0000 7fbf970e4700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fbf90076d00 con 0x7fbf90107890 2024-05-03T18:37:47.724 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:40.750+0000 7fbf94e80700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf90107890 0x7fbf90107cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:47.724 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:40.750+0000 7fbf94e80700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf90107890 0x7fbf90107cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38812/0 (socket says 172.21.15.32:38812) 2024-05-03T18:37:47.724 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:40.750+0000 7fbf94e80700 1 -- 172.21.15.32:0/374191075 learned_addr learned my addr 172.21.15.32:0/374191075 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:37:47.724 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:40.751+0000 7fbf94e80700 1 -- 172.21.15.32:0/374191075 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf901081f0 con 0x7fbf90107890 2024-05-03T18:37:47.724 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:40.751+0000 7fbf94e80700 1 --2- 172.21.15.32:0/374191075 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf90107890 0x7fbf90107cb0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fbf840098a0 tx=0x7fbf84009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a00033145a34b117 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:37:47.724 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:40.751+0000 7fbf8f7fe700 1 -- 172.21.15.32:0/374191075 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbf84004160 con 0x7fbf90107890 2024-05-03T18:37:47.724 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:40.751+0000 7fbf8f7fe700 1 -- 172.21.15.32:0/374191075 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf84004300 con 0x7fbf90107890 2024-05-03T18:37:47.724 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:40.751+0000 7fbf8f7fe700 1 -- 172.21.15.32:0/374191075 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbf84011a20 con 0x7fbf90107890 2024-05-03T18:37:47.724 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:40.752+0000 7fbf970e4700 1 -- 172.21.15.32:0/374191075 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf90107890 msgr2=0x7fbf90107cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:37:47.724 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:40.752+0000 7fbf970e4700 1 --2- 172.21.15.32:0/374191075 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf90107890 0x7fbf90107cb0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fbf840098a0 tx=0x7fbf84009bb0 comp rx=0 tx=0).stop 2024-05-03T18:37:47.724 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:40.752+0000 7fbf970e4700 1 -- 172.21.15.32:0/374191075 shutdown_connections 2024-05-03T18:37:47.724 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:40.752+0000 7fbf970e4700 1 --2- 172.21.15.32:0/374191075 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf90107890 0x7fbf90107cb0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:47.725 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:40.752+0000 7fbf970e4700 1 -- 172.21.15.32:0/374191075 >> 172.21.15.32:0/374191075 conn(0x7fbf901034d0 msgr2=0x7fbf901058f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:37:47.725 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:40.752+0000 7fbf970e4700 1 -- 172.21.15.32:0/374191075 shutdown_connections 2024-05-03T18:37:47.725 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:40.752+0000 7fbf970e4700 1 -- 172.21.15.32:0/374191075 wait complete. 2024-05-03T18:37:47.725 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:40.753+0000 7fbf970e4700 1 Processor -- start 2024-05-03T18:37:47.725 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:40.753+0000 7fbf970e4700 1 -- start start 2024-05-03T18:37:47.725 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:40.753+0000 7fbf970e4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf90107890 0x7fbf9019f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:47.725 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:40.753+0000 7fbf970e4700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fbf90109260 con 0x7fbf90107890 2024-05-03T18:37:47.725 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:40.753+0000 7fbf94e80700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf90107890 0x7fbf9019f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:47.725 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:40.753+0000 7fbf94e80700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf90107890 0x7fbf9019f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38818/0 (socket says 172.21.15.32:38818) 2024-05-03T18:37:47.725 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:40.753+0000 7fbf94e80700 1 -- 172.21.15.32:0/2216794815 learned_addr learned my addr 172.21.15.32:0/2216794815 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:37:47.725 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:40.753+0000 7fbf94e80700 1 -- 172.21.15.32:0/2216794815 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf9019f5c0 con 0x7fbf90107890 2024-05-03T18:37:47.725 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:40.753+0000 7fbf94e80700 1 --2- 172.21.15.32:0/2216794815 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf90107890 0x7fbf9019f080 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fbf84000c00 tx=0x7fbf84011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:37:47.725 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:40.754+0000 7fbf8dffb700 1 -- 172.21.15.32:0/2216794815 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbf84003aa0 con 0x7fbf90107890 2024-05-03T18:37:47.725 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:40.754+0000 7fbf8dffb700 1 -- 172.21.15.32:0/2216794815 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf8401b8f0 con 0x7fbf90107890 2024-05-03T18:37:47.726 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:40.754+0000 7fbf970e4700 1 -- 172.21.15.32:0/2216794815 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf9019f850 con 0x7fbf90107890 2024-05-03T18:37:47.726 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:40.754+0000 7fbf8dffb700 1 -- 172.21.15.32:0/2216794815 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbf84023870 con 0x7fbf90107890 2024-05-03T18:37:47.726 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:40.754+0000 7fbf970e4700 1 -- 172.21.15.32:0/2216794815 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf901a2540 con 0x7fbf90107890 2024-05-03T18:37:47.726 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:40.755+0000 7fbf8dffb700 1 -- 172.21.15.32:0/2216794815 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 9) ==== 50235+0+0 (secure 0 0 0) 0x7fbf84023b70 con 0x7fbf90107890 2024-05-03T18:37:47.726 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:40.755+0000 7fbf8dffb700 1 --2- 172.21.15.32:0/2216794815 >> [v2:172.21.15.32:6800/2019702098,v1:172.21.15.32:6801/2019702098] conn(0x7fbf8003e4c0 0x7fbf80040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:47.726 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:40.755+0000 7fbf8dffb700 1 -- 172.21.15.32:0/2216794815 --> [v2:172.21.15.32:6800/2019702098,v1:172.21.15.32:6801/2019702098] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fbf80041090 con 0x7fbf8003e4c0 2024-05-03T18:37:47.726 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:40.755+0000 7fbf8ffff700 1 -- 172.21.15.32:0/2216794815 >> [v2:172.21.15.32:6800/2019702098,v1:172.21.15.32:6801/2019702098] conn(0x7fbf8003e4c0 msgr2=0x7fbf80040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6800/2019702098 2024-05-03T18:37:47.726 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:40.755+0000 7fbf8dffb700 1 -- 172.21.15.32:0/2216794815 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fbf84053e40 con 0x7fbf90107890 2024-05-03T18:37:47.726 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:40.755+0000 7fbf8ffff700 1 --2- 172.21.15.32:0/2216794815 >> [v2:172.21.15.32:6800/2019702098,v1:172.21.15.32:6801/2019702098] conn(0x7fbf8003e4c0 0x7fbf80040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-03T18:37:47.726 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:40.956+0000 7fbf8ffff700 1 -- 172.21.15.32:0/2216794815 >> [v2:172.21.15.32:6800/2019702098,v1:172.21.15.32:6801/2019702098] conn(0x7fbf8003e4c0 msgr2=0x7fbf80040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6800/2019702098 2024-05-03T18:37:47.726 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:40.956+0000 7fbf8ffff700 1 --2- 172.21.15.32:0/2216794815 >> [v2:172.21.15.32:6800/2019702098,v1:172.21.15.32:6801/2019702098] conn(0x7fbf8003e4c0 0x7fbf80040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-03T18:37:47.726 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:41.357+0000 7fbf8ffff700 1 -- 172.21.15.32:0/2216794815 >> [v2:172.21.15.32:6800/2019702098,v1:172.21.15.32:6801/2019702098] conn(0x7fbf8003e4c0 msgr2=0x7fbf80040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6800/2019702098 2024-05-03T18:37:47.727 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:41.357+0000 7fbf8ffff700 1 --2- 172.21.15.32:0/2216794815 >> [v2:172.21.15.32:6800/2019702098,v1:172.21.15.32:6801/2019702098] conn(0x7fbf8003e4c0 0x7fbf80040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-03T18:37:47.727 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:42.158+0000 7fbf8ffff700 1 -- 172.21.15.32:0/2216794815 >> [v2:172.21.15.32:6800/2019702098,v1:172.21.15.32:6801/2019702098] conn(0x7fbf8003e4c0 msgr2=0x7fbf80040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6800/2019702098 2024-05-03T18:37:47.727 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:42.158+0000 7fbf8ffff700 1 --2- 172.21.15.32:0/2216794815 >> [v2:172.21.15.32:6800/2019702098,v1:172.21.15.32:6801/2019702098] conn(0x7fbf8003e4c0 0x7fbf80040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-03T18:37:47.727 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:43.759+0000 7fbf8ffff700 1 -- 172.21.15.32:0/2216794815 >> [v2:172.21.15.32:6800/2019702098,v1:172.21.15.32:6801/2019702098] conn(0x7fbf8003e4c0 msgr2=0x7fbf80040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6800/2019702098 2024-05-03T18:37:47.727 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:43.759+0000 7fbf8ffff700 1 --2- 172.21.15.32:0/2216794815 >> [v2:172.21.15.32:6800/2019702098,v1:172.21.15.32:6801/2019702098] conn(0x7fbf8003e4c0 0x7fbf80040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-05-03T18:37:47.727 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:45.881+0000 7fbf8dffb700 1 -- 172.21.15.32:0/2216794815 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mgrmap(e 10) ==== 50002+0+0 (secure 0 0 0) 0x7fbf8402c3f0 con 0x7fbf90107890 2024-05-03T18:37:47.727 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:45.881+0000 7fbf8dffb700 1 -- 172.21.15.32:0/2216794815 >> [v2:172.21.15.32:6800/2019702098,v1:172.21.15.32:6801/2019702098] conn(0x7fbf8003e4c0 msgr2=0x7fbf80040980 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-03T18:37:47.727 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:45.881+0000 7fbf8dffb700 1 --2- 172.21.15.32:0/2216794815 >> [v2:172.21.15.32:6800/2019702098,v1:172.21.15.32:6801/2019702098] conn(0x7fbf8003e4c0 0x7fbf80040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:47.727 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:46.882+0000 7fbf8dffb700 1 -- 172.21.15.32:0/2216794815 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mgrmap(e 11) ==== 50129+0+0 (secure 0 0 0) 0x7fbf84055230 con 0x7fbf90107890 2024-05-03T18:37:47.727 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:46.882+0000 7fbf8dffb700 1 --2- 172.21.15.32:0/2216794815 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fbf80041f40 0x7fbf80044330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:47.727 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:46.882+0000 7fbf8dffb700 1 -- 172.21.15.32:0/2216794815 --> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fbf840523c0 con 0x7fbf80041f40 2024-05-03T18:37:47.727 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:46.883+0000 7fbf8ffff700 1 --2- 172.21.15.32:0/2216794815 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fbf80041f40 0x7fbf80044330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:47.728 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:46.884+0000 7fbf8ffff700 1 --2- 172.21.15.32:0/2216794815 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fbf80041f40 0x7fbf80044330 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fbf7c003a10 tx=0x7fbf7c0092b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:37:47.728 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:46.885+0000 7fbf8dffb700 1 -- 172.21.15.32:0/2216794815 <== mgr.14152 v2:172.21.15.32:6800/1796547904 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7fbf840523c0 con 0x7fbf80041f40 2024-05-03T18:37:47.728 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:46.893+0000 7fbf970e4700 1 -- 172.21.15.32:0/2216794815 --> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7fbf90064b00 con 0x7fbf80041f40 2024-05-03T18:37:47.728 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:46.894+0000 7fbf8dffb700 1 -- 172.21.15.32:0/2216794815 <== mgr.14152 v2:172.21.15.32:6800/1796547904 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7fbf90064b00 con 0x7fbf80041f40 2024-05-03T18:37:47.728 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:46.894+0000 7fbf970e4700 1 -- 172.21.15.32:0/2216794815 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fbf80041f40 msgr2=0x7fbf80044330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:37:47.728 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:46.894+0000 7fbf970e4700 1 --2- 172.21.15.32:0/2216794815 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fbf80041f40 0x7fbf80044330 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fbf7c003a10 tx=0x7fbf7c0092b0 comp rx=0 tx=0).stop 2024-05-03T18:37:47.728 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:46.894+0000 7fbf970e4700 1 -- 172.21.15.32:0/2216794815 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf90107890 msgr2=0x7fbf9019f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:37:47.728 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:46.894+0000 7fbf970e4700 1 --2- 172.21.15.32:0/2216794815 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf90107890 0x7fbf9019f080 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fbf84000c00 tx=0x7fbf84011aa0 comp rx=0 tx=0).stop 2024-05-03T18:37:47.728 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:46.894+0000 7fbf970e4700 1 -- 172.21.15.32:0/2216794815 shutdown_connections 2024-05-03T18:37:47.728 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:46.894+0000 7fbf970e4700 1 --2- 172.21.15.32:0/2216794815 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf90107890 0x7fbf9019f080 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:47.728 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:46.894+0000 7fbf970e4700 1 --2- 172.21.15.32:0/2216794815 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fbf80041f40 0x7fbf80044330 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:47.728 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:46.894+0000 7fbf970e4700 1 --2- 172.21.15.32:0/2216794815 >> [v2:172.21.15.32:6800/2019702098,v1:172.21.15.32:6801/2019702098] conn(0x7fbf8003e4c0 0x7fbf80040980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:47.728 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:46.894+0000 7fbf970e4700 1 -- 172.21.15.32:0/2216794815 >> 172.21.15.32:0/2216794815 conn(0x7fbf901034d0 msgr2=0x7fbf90195ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:37:47.728 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:46.894+0000 7fbf970e4700 1 -- 172.21.15.32:0/2216794815 shutdown_connections 2024-05-03T18:37:47.729 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:46.894+0000 7fbf970e4700 1 -- 172.21.15.32:0/2216794815 wait complete. 2024-05-03T18:37:47.729 INFO:teuthology.orchestra.run.smithi032.stdout:mgr epoch 9 is available 2024-05-03T18:37:47.729 INFO:teuthology.orchestra.run.smithi032.stdout:Generating a dashboard self-signed certificate... 2024-05-03T18:37:48.040 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:37:47 smithi032 ceph-mon[29763]: [03/May/2024:18:37:46] ENGINE Serving on https://172.21.15.32:7150 2024-05-03T18:37:48.040 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:37:47 smithi032 ceph-mon[29763]: [03/May/2024:18:37:46] ENGINE Bus STARTED 2024-05-03T18:37:48.040 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:37:47 smithi032 ceph-mon[29763]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-05-03T18:37:48.040 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:37:47 smithi032 ceph-mon[29763]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-05-03T18:37:49.086 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-05-03T18:37:49.086 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:48.149+0000 7fb558817700 1 Processor -- start 2024-05-03T18:37:49.087 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:48.149+0000 7fb558817700 1 -- start start 2024-05-03T18:37:49.087 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:48.150+0000 7fb558817700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb554107e90 0x7fb5541082b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:49.087 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:48.150+0000 7fb558817700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fb5541087f0 con 0x7fb554107e90 2024-05-03T18:37:49.087 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:48.150+0000 7fb55259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb554107e90 0x7fb5541082b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:49.087 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:48.150+0000 7fb55259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb554107e90 0x7fb5541082b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58326/0 (socket says 172.21.15.32:58326) 2024-05-03T18:37:49.087 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:48.150+0000 7fb55259c700 1 -- 172.21.15.32:0/2603697379 learned_addr learned my addr 172.21.15.32:0/2603697379 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:37:49.087 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:48.150+0000 7fb55259c700 1 -- 172.21.15.32:0/2603697379 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb554108ff0 con 0x7fb554107e90 2024-05-03T18:37:49.087 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:48.151+0000 7fb55259c700 1 --2- 172.21.15.32:0/2603697379 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb554107e90 0x7fb5541082b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb5440098a0 tx=0x7fb544009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a4b16bd46584a5f server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:37:49.087 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:48.151+0000 7fb55159a700 1 -- 172.21.15.32:0/2603697379 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb544004160 con 0x7fb554107e90 2024-05-03T18:37:49.087 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:48.151+0000 7fb55159a700 1 -- 172.21.15.32:0/2603697379 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb544004300 con 0x7fb554107e90 2024-05-03T18:37:49.087 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:48.151+0000 7fb55159a700 1 -- 172.21.15.32:0/2603697379 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb544011a20 con 0x7fb554107e90 2024-05-03T18:37:49.087 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:48.151+0000 7fb558817700 1 -- 172.21.15.32:0/2603697379 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb554107e90 msgr2=0x7fb5541082b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:37:49.088 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:48.151+0000 7fb558817700 1 --2- 172.21.15.32:0/2603697379 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb554107e90 0x7fb5541082b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb5440098a0 tx=0x7fb544009bb0 comp rx=0 tx=0).stop 2024-05-03T18:37:49.088 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:48.151+0000 7fb558817700 1 -- 172.21.15.32:0/2603697379 shutdown_connections 2024-05-03T18:37:49.088 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:48.151+0000 7fb558817700 1 --2- 172.21.15.32:0/2603697379 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb554107e90 0x7fb5541082b0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:49.088 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:48.152+0000 7fb558817700 1 -- 172.21.15.32:0/2603697379 >> 172.21.15.32:0/2603697379 conn(0x7fb5541034d0 msgr2=0x7fb5541058f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:37:49.088 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:48.152+0000 7fb558817700 1 -- 172.21.15.32:0/2603697379 shutdown_connections 2024-05-03T18:37:49.088 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:48.152+0000 7fb558817700 1 -- 172.21.15.32:0/2603697379 wait complete. 2024-05-03T18:37:49.088 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:48.152+0000 7fb558817700 1 Processor -- start 2024-05-03T18:37:49.088 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:48.152+0000 7fb558817700 1 -- start start 2024-05-03T18:37:49.088 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:48.153+0000 7fb558817700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb554107e90 0x7fb5540769a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:49.088 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:48.153+0000 7fb558817700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fb55410c430 con 0x7fb554107e90 2024-05-03T18:37:49.088 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:48.153+0000 7fb55259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb554107e90 0x7fb5540769a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:49.088 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:48.153+0000 7fb55259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb554107e90 0x7fb5540769a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58334/0 (socket says 172.21.15.32:58334) 2024-05-03T18:37:49.088 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:48.153+0000 7fb55259c700 1 -- 172.21.15.32:0/910590207 learned_addr learned my addr 172.21.15.32:0/910590207 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:37:49.088 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:48.153+0000 7fb55259c700 1 -- 172.21.15.32:0/910590207 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb55419fb60 con 0x7fb554107e90 2024-05-03T18:37:49.089 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:48.153+0000 7fb55259c700 1 --2- 172.21.15.32:0/910590207 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb554107e90 0x7fb5540769a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb5440116e0 tx=0x7fb544011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:37:49.089 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:48.153+0000 7fb5437fe700 1 -- 172.21.15.32:0/910590207 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb544003a60 con 0x7fb554107e90 2024-05-03T18:37:49.089 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:48.153+0000 7fb5437fe700 1 -- 172.21.15.32:0/910590207 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb54401b8f0 con 0x7fb554107e90 2024-05-03T18:37:49.089 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:48.153+0000 7fb558817700 1 -- 172.21.15.32:0/910590207 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb55419fd30 con 0x7fb554107e90 2024-05-03T18:37:49.089 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:48.153+0000 7fb5437fe700 1 -- 172.21.15.32:0/910590207 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb544019860 con 0x7fb554107e90 2024-05-03T18:37:49.089 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:48.153+0000 7fb558817700 1 -- 172.21.15.32:0/910590207 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5541a0150 con 0x7fb554107e90 2024-05-03T18:37:49.089 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:48.154+0000 7fb5437fe700 1 -- 172.21.15.32:0/910590207 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 12) ==== 50235+0+0 (secure 0 0 0) 0x7fb544019b10 con 0x7fb554107e90 2024-05-03T18:37:49.089 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:48.154+0000 7fb558817700 1 -- 172.21.15.32:0/910590207 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb55410c5b0 con 0x7fb554107e90 2024-05-03T18:37:49.089 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:48.154+0000 7fb5437fe700 1 --2- 172.21.15.32:0/910590207 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fb53c03e4d0 0x7fb53c040990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:49.089 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:48.154+0000 7fb5437fe700 1 -- 172.21.15.32:0/910590207 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fb544053f40 con 0x7fb554107e90 2024-05-03T18:37:49.089 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:48.155+0000 7fb551d9b700 1 --2- 172.21.15.32:0/910590207 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fb53c03e4d0 0x7fb53c040990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:49.089 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:48.155+0000 7fb551d9b700 1 --2- 172.21.15.32:0/910590207 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fb53c03e4d0 0x7fb53c040990 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb548006fd0 tx=0x7fb548006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:37:49.089 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:48.160+0000 7fb5437fe700 1 -- 172.21.15.32:0/910590207 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fb54401f030 con 0x7fb554107e90 2024-05-03T18:37:49.089 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:48.405+0000 7fb558817700 1 -- 172.21.15.32:0/910590207 --> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7fb5541046d0 con 0x7fb53c03e4d0 2024-05-03T18:37:49.090 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:48.535+0000 7fb5437fe700 1 -- 172.21.15.32:0/910590207 <== mgr.14152 v2:172.21.15.32:6800/1796547904 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7fb5541046d0 con 0x7fb53c03e4d0 2024-05-03T18:37:49.090 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:48.537+0000 7fb558817700 1 -- 172.21.15.32:0/910590207 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fb53c03e4d0 msgr2=0x7fb53c040990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:37:49.090 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:48.537+0000 7fb558817700 1 --2- 172.21.15.32:0/910590207 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fb53c03e4d0 0x7fb53c040990 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb548006fd0 tx=0x7fb548006e40 comp rx=0 tx=0).stop 2024-05-03T18:37:49.090 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:48.537+0000 7fb558817700 1 -- 172.21.15.32:0/910590207 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb554107e90 msgr2=0x7fb5540769a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:37:49.090 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:48.537+0000 7fb558817700 1 --2- 172.21.15.32:0/910590207 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb554107e90 0x7fb5540769a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb5440116e0 tx=0x7fb544011aa0 comp rx=0 tx=0).stop 2024-05-03T18:37:49.090 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:48.537+0000 7fb558817700 1 -- 172.21.15.32:0/910590207 shutdown_connections 2024-05-03T18:37:49.090 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:48.537+0000 7fb558817700 1 --2- 172.21.15.32:0/910590207 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fb53c03e4d0 0x7fb53c040990 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:49.090 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:48.537+0000 7fb558817700 1 --2- 172.21.15.32:0/910590207 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb554107e90 0x7fb5540769a0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:49.090 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:48.537+0000 7fb558817700 1 -- 172.21.15.32:0/910590207 >> 172.21.15.32:0/910590207 conn(0x7fb5541034d0 msgr2=0x7fb554103fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:37:49.090 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:48.537+0000 7fb558817700 1 -- 172.21.15.32:0/910590207 shutdown_connections 2024-05-03T18:37:49.090 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:48.537+0000 7fb558817700 1 -- 172.21.15.32:0/910590207 wait complete. 2024-05-03T18:37:49.090 INFO:teuthology.orchestra.run.smithi032.stdout:Creating initial admin user... 2024-05-03T18:37:49.205 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:37:48 smithi032 ceph-mon[29763]: mgrmap e12: x(active, since 2s) 2024-05-03T18:37:49.205 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:37:48 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:37:49.205 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:37:48 smithi032 ceph-mon[29763]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:37:49.205 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:37:48 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:37:49.205 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:37:48 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:37:50.599 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$NR7PcP2pFs/fTNc8tjtPVeRnvQQoVUu3nfuNX0fS6Ct8figPAF4Iq", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1714761470, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-05-03T18:37:50.599 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:49.546+0000 7f387e04f700 1 Processor -- start 2024-05-03T18:37:50.600 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:49.546+0000 7f387e04f700 1 -- start start 2024-05-03T18:37:50.600 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:49.546+0000 7f387e04f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3878109a40 0x7f387810be30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:50.600 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:49.546+0000 7f387e04f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f3878076d60 con 0x7f3878109a40 2024-05-03T18:37:50.600 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:49.546+0000 7f38777fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3878109a40 0x7f387810be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:50.600 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:49.546+0000 7f38777fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3878109a40 0x7f387810be30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58344/0 (socket says 172.21.15.32:58344) 2024-05-03T18:37:50.600 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:49.546+0000 7f38777fe700 1 -- 172.21.15.32:0/3043907839 learned_addr learned my addr 172.21.15.32:0/3043907839 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:37:50.600 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:49.546+0000 7f38777fe700 1 -- 172.21.15.32:0/3043907839 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f387810c370 con 0x7f3878109a40 2024-05-03T18:37:50.600 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:49.547+0000 7f38777fe700 1 --2- 172.21.15.32:0/3043907839 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3878109a40 0x7f387810be30 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f38680098a0 tx=0x7f3868009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9520cd30e47dc7ad server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:37:50.600 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:49.547+0000 7f38767fc700 1 -- 172.21.15.32:0/3043907839 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3868004160 con 0x7f3878109a40 2024-05-03T18:37:50.600 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:49.547+0000 7f38767fc700 1 -- 172.21.15.32:0/3043907839 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3868004300 con 0x7f3878109a40 2024-05-03T18:37:50.600 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:49.547+0000 7f38767fc700 1 -- 172.21.15.32:0/3043907839 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3868011a20 con 0x7f3878109a40 2024-05-03T18:37:50.600 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:49.548+0000 7f387e04f700 1 -- 172.21.15.32:0/3043907839 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3878109a40 msgr2=0x7f387810be30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:37:50.600 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:49.548+0000 7f387e04f700 1 --2- 172.21.15.32:0/3043907839 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3878109a40 0x7f387810be30 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f38680098a0 tx=0x7f3868009bb0 comp rx=0 tx=0).stop 2024-05-03T18:37:50.601 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:49.548+0000 7f387e04f700 1 -- 172.21.15.32:0/3043907839 shutdown_connections 2024-05-03T18:37:50.601 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:49.548+0000 7f387e04f700 1 --2- 172.21.15.32:0/3043907839 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3878109a40 0x7f387810be30 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:50.601 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:49.548+0000 7f387e04f700 1 -- 172.21.15.32:0/3043907839 >> 172.21.15.32:0/3043907839 conn(0x7f38781034d0 msgr2=0x7f38781058f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:37:50.601 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:49.548+0000 7f387e04f700 1 -- 172.21.15.32:0/3043907839 shutdown_connections 2024-05-03T18:37:50.601 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:49.548+0000 7f387e04f700 1 -- 172.21.15.32:0/3043907839 wait complete. 2024-05-03T18:37:50.601 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:49.548+0000 7f387e04f700 1 Processor -- start 2024-05-03T18:37:50.601 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:49.548+0000 7f387e04f700 1 -- start start 2024-05-03T18:37:50.601 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:49.549+0000 7f387e04f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3878109a40 0x7f38781092c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:50.601 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:49.549+0000 7f387e04f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f387810d370 con 0x7f3878109a40 2024-05-03T18:37:50.601 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:49.549+0000 7f38777fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3878109a40 0x7f38781092c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:50.601 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:49.549+0000 7f38777fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3878109a40 0x7f38781092c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58346/0 (socket says 172.21.15.32:58346) 2024-05-03T18:37:50.601 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:49.549+0000 7f38777fe700 1 -- 172.21.15.32:0/1902563998 learned_addr learned my addr 172.21.15.32:0/1902563998 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:37:50.601 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:49.549+0000 7f38777fe700 1 -- 172.21.15.32:0/1902563998 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3878109800 con 0x7f3878109a40 2024-05-03T18:37:50.601 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:49.549+0000 7f38777fe700 1 --2- 172.21.15.32:0/1902563998 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3878109a40 0x7f38781092c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3868000c00 tx=0x7f3868011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:37:50.602 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:49.549+0000 7f3874ff9700 1 -- 172.21.15.32:0/1902563998 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3868003aa0 con 0x7f3878109a40 2024-05-03T18:37:50.602 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:49.549+0000 7f3874ff9700 1 -- 172.21.15.32:0/1902563998 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f386801b8f0 con 0x7f3878109a40 2024-05-03T18:37:50.602 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:49.549+0000 7f387e04f700 1 -- 172.21.15.32:0/1902563998 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38781079f0 con 0x7f3878109a40 2024-05-03T18:37:50.602 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:49.549+0000 7f387e04f700 1 -- 172.21.15.32:0/1902563998 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3878107f30 con 0x7f3878109a40 2024-05-03T18:37:50.602 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:49.550+0000 7f3874ff9700 1 -- 172.21.15.32:0/1902563998 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3868023870 con 0x7f3878109a40 2024-05-03T18:37:50.602 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:49.550+0000 7f3874ff9700 1 -- 172.21.15.32:0/1902563998 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 12) ==== 50235+0+0 (secure 0 0 0) 0x7f3868023b20 con 0x7f3878109a40 2024-05-03T18:37:50.602 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:49.551+0000 7f3874ff9700 1 --2- 172.21.15.32:0/1902563998 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f386003e570 0x7f3860040a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:50.602 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:49.551+0000 7f3874ff9700 1 -- 172.21.15.32:0/1902563998 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f3868054ef0 con 0x7f3878109a40 2024-05-03T18:37:50.602 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:49.551+0000 7f3876ffd700 1 --2- 172.21.15.32:0/1902563998 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f386003e570 0x7f3860040a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:50.602 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:49.551+0000 7f387e04f700 1 -- 172.21.15.32:0/1902563998 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3864005360 con 0x7f3878109a40 2024-05-03T18:37:50.602 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:49.551+0000 7f3876ffd700 1 --2- 172.21.15.32:0/1902563998 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f386003e570 0x7f3860040a30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f386c006fd0 tx=0x7f386c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:37:50.602 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:49.557+0000 7f3874ff9700 1 -- 172.21.15.32:0/1902563998 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f386801f030 con 0x7f3878109a40 2024-05-03T18:37:50.602 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:49.788+0000 7f387e04f700 1 -- 172.21.15.32:0/1902563998 --> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7f3864002430 con 0x7f386003e570 2024-05-03T18:37:50.602 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:50.035+0000 7f3874ff9700 1 -- 172.21.15.32:0/1902563998 <== mgr.14152 v2:172.21.15.32:6800/1796547904 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7f3864002430 con 0x7f386003e570 2024-05-03T18:37:50.603 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:50.037+0000 7f387e04f700 1 -- 172.21.15.32:0/1902563998 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f386003e570 msgr2=0x7f3860040a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:37:50.603 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:50.037+0000 7f387e04f700 1 --2- 172.21.15.32:0/1902563998 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f386003e570 0x7f3860040a30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f386c006fd0 tx=0x7f386c006e40 comp rx=0 tx=0).stop 2024-05-03T18:37:50.603 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:50.038+0000 7f387e04f700 1 -- 172.21.15.32:0/1902563998 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3878109a40 msgr2=0x7f38781092c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:37:50.603 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:50.038+0000 7f387e04f700 1 --2- 172.21.15.32:0/1902563998 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3878109a40 0x7f38781092c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3868000c00 tx=0x7f3868011aa0 comp rx=0 tx=0).stop 2024-05-03T18:37:50.603 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:50.038+0000 7f387e04f700 1 -- 172.21.15.32:0/1902563998 shutdown_connections 2024-05-03T18:37:50.603 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:50.038+0000 7f387e04f700 1 --2- 172.21.15.32:0/1902563998 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f386003e570 0x7f3860040a30 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:50.603 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:50.038+0000 7f387e04f700 1 --2- 172.21.15.32:0/1902563998 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3878109a40 0x7f38781092c0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:50.603 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:50.038+0000 7f387e04f700 1 -- 172.21.15.32:0/1902563998 >> 172.21.15.32:0/1902563998 conn(0x7f38781034d0 msgr2=0x7f3878103f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:37:50.603 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:50.038+0000 7f387e04f700 1 -- 172.21.15.32:0/1902563998 shutdown_connections 2024-05-03T18:37:50.603 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:50.038+0000 7f387e04f700 1 -- 172.21.15.32:0/1902563998 wait complete. 2024-05-03T18:37:50.603 INFO:teuthology.orchestra.run.smithi032.stdout:Fetching dashboard port number... 2024-05-03T18:37:50.928 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:37:50 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:37:50.928 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:37:50 smithi032 ceph-mon[29763]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:37:50.928 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:37:50 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:37:51.817 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:37:51 smithi032 ceph-mon[29763]: from='client.? 172.21.15.32:0/1540413897' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-05-03T18:37:52.480 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stdout 8443 2024-05-03T18:37:52.481 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:51.039+0000 7fdfcb361700 1 Processor -- start 2024-05-03T18:37:52.481 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:51.039+0000 7fdfcb361700 1 -- start start 2024-05-03T18:37:52.481 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:51.039+0000 7fdfcb361700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdfc4108fa0 0x7fdfc41093c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:52.481 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:51.039+0000 7fdfcb361700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fdfc4109990 con 0x7fdfc4108fa0 2024-05-03T18:37:52.481 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:51.039+0000 7fdfc90fd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdfc4108fa0 0x7fdfc41093c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:52.481 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:51.039+0000 7fdfc90fd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdfc4108fa0 0x7fdfc41093c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58350/0 (socket says 172.21.15.32:58350) 2024-05-03T18:37:52.481 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:51.039+0000 7fdfc90fd700 1 -- 172.21.15.32:0/626594806 learned_addr learned my addr 172.21.15.32:0/626594806 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:37:52.481 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:51.039+0000 7fdfc90fd700 1 -- 172.21.15.32:0/626594806 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfc410a1a0 con 0x7fdfc4108fa0 2024-05-03T18:37:52.481 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:51.040+0000 7fdfc90fd700 1 --2- 172.21.15.32:0/626594806 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdfc4108fa0 0x7fdfc41093c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fdfc00098a0 tx=0x7fdfc0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7339500fa2bf21c server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:37:52.481 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:51.040+0000 7fdfbbfff700 1 -- 172.21.15.32:0/626594806 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdfc0004160 con 0x7fdfc4108fa0 2024-05-03T18:37:52.481 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:51.040+0000 7fdfbbfff700 1 -- 172.21.15.32:0/626594806 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdfc0004300 con 0x7fdfc4108fa0 2024-05-03T18:37:52.482 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:51.040+0000 7fdfbbfff700 1 -- 172.21.15.32:0/626594806 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdfc0011a20 con 0x7fdfc4108fa0 2024-05-03T18:37:52.482 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:51.041+0000 7fdfcb361700 1 -- 172.21.15.32:0/626594806 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdfc4108fa0 msgr2=0x7fdfc41093c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:37:52.482 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:51.041+0000 7fdfcb361700 1 --2- 172.21.15.32:0/626594806 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdfc4108fa0 0x7fdfc41093c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fdfc00098a0 tx=0x7fdfc0009bb0 comp rx=0 tx=0).stop 2024-05-03T18:37:52.482 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:51.041+0000 7fdfcb361700 1 -- 172.21.15.32:0/626594806 shutdown_connections 2024-05-03T18:37:52.482 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:51.041+0000 7fdfcb361700 1 --2- 172.21.15.32:0/626594806 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdfc4108fa0 0x7fdfc41093c0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:52.482 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:51.041+0000 7fdfcb361700 1 -- 172.21.15.32:0/626594806 >> 172.21.15.32:0/626594806 conn(0x7fdfc4104580 msgr2=0x7fdfc41069a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:37:52.482 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:51.041+0000 7fdfcb361700 1 -- 172.21.15.32:0/626594806 shutdown_connections 2024-05-03T18:37:52.482 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:51.041+0000 7fdfcb361700 1 -- 172.21.15.32:0/626594806 wait complete. 2024-05-03T18:37:52.482 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:51.041+0000 7fdfcb361700 1 Processor -- start 2024-05-03T18:37:52.482 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:51.042+0000 7fdfcb361700 1 -- start start 2024-05-03T18:37:52.482 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:51.042+0000 7fdfcb361700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdfc4108fa0 0x7fdfc407eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:52.482 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:51.042+0000 7fdfcb361700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fdfc410d550 con 0x7fdfc4108fa0 2024-05-03T18:37:52.482 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:51.042+0000 7fdfc90fd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdfc4108fa0 0x7fdfc407eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:52.482 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:51.042+0000 7fdfc90fd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdfc4108fa0 0x7fdfc407eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58360/0 (socket says 172.21.15.32:58360) 2024-05-03T18:37:52.483 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:51.042+0000 7fdfc90fd700 1 -- 172.21.15.32:0/1540413897 learned_addr learned my addr 172.21.15.32:0/1540413897 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:37:52.483 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:51.042+0000 7fdfc90fd700 1 -- 172.21.15.32:0/1540413897 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfc407f0c0 con 0x7fdfc4108fa0 2024-05-03T18:37:52.483 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:51.042+0000 7fdfc90fd700 1 --2- 172.21.15.32:0/1540413897 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdfc4108fa0 0x7fdfc407eb80 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fdfc0000c00 tx=0x7fdfc0011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:37:52.483 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:51.043+0000 7fdfba7fc700 1 -- 172.21.15.32:0/1540413897 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdfc0003aa0 con 0x7fdfc4108fa0 2024-05-03T18:37:52.483 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:51.043+0000 7fdfcb361700 1 -- 172.21.15.32:0/1540413897 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfc407d2b0 con 0x7fdfc4108fa0 2024-05-03T18:37:52.483 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:51.043+0000 7fdfba7fc700 1 -- 172.21.15.32:0/1540413897 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdfc001b8f0 con 0x7fdfc4108fa0 2024-05-03T18:37:52.483 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:51.044+0000 7fdfba7fc700 1 -- 172.21.15.32:0/1540413897 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdfc00238a0 con 0x7fdfc4108fa0 2024-05-03T18:37:52.483 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:51.044+0000 7fdfcb361700 1 -- 172.21.15.32:0/1540413897 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfc407d7f0 con 0x7fdfc4108fa0 2024-05-03T18:37:52.483 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:51.044+0000 7fdfba7fc700 1 -- 172.21.15.32:0/1540413897 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 12) ==== 50235+0+0 (secure 0 0 0) 0x7fdfc0023a40 con 0x7fdfc4108fa0 2024-05-03T18:37:52.483 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:51.044+0000 7fdfba7fc700 1 --2- 172.21.15.32:0/1540413897 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fdfb003e4c0 0x7fdfb0040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:52.483 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:51.044+0000 7fdfba7fc700 1 -- 172.21.15.32:0/1540413897 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fdfc00557f0 con 0x7fdfc4108fa0 2024-05-03T18:37:52.483 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:51.044+0000 7fdfc88fc700 1 --2- 172.21.15.32:0/1540413897 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fdfb003e4c0 0x7fdfb0040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:52.483 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:51.045+0000 7fdfcb361700 1 -- 172.21.15.32:0/1540413897 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfc4064b00 con 0x7fdfc4108fa0 2024-05-03T18:37:52.483 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:51.046+0000 7fdfc88fc700 1 --2- 172.21.15.32:0/1540413897 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fdfb003e4c0 0x7fdfb0040980 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fdfb4006fd0 tx=0x7fdfb4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:37:52.484 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:51.055+0000 7fdfba7fc700 1 -- 172.21.15.32:0/1540413897 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fdfc0058070 con 0x7fdfc4108fa0 2024-05-03T18:37:52.484 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:51.267+0000 7fdfcb361700 1 -- 172.21.15.32:0/1540413897 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7fdfc407da70 con 0x7fdfc4108fa0 2024-05-03T18:37:52.484 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:51.268+0000 7fdfba7fc700 1 -- 172.21.15.32:0/1540413897 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7fdfc001f030 con 0x7fdfc4108fa0 2024-05-03T18:37:52.484 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:51.270+0000 7fdfcb361700 1 -- 172.21.15.32:0/1540413897 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fdfb003e4c0 msgr2=0x7fdfb0040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:37:52.484 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:51.270+0000 7fdfcb361700 1 --2- 172.21.15.32:0/1540413897 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fdfb003e4c0 0x7fdfb0040980 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fdfb4006fd0 tx=0x7fdfb4006e40 comp rx=0 tx=0).stop 2024-05-03T18:37:52.484 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:51.270+0000 7fdfcb361700 1 -- 172.21.15.32:0/1540413897 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdfc4108fa0 msgr2=0x7fdfc407eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:37:52.484 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:51.270+0000 7fdfcb361700 1 --2- 172.21.15.32:0/1540413897 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdfc4108fa0 0x7fdfc407eb80 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fdfc0000c00 tx=0x7fdfc0011aa0 comp rx=0 tx=0).stop 2024-05-03T18:37:52.484 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:51.270+0000 7fdfcb361700 1 -- 172.21.15.32:0/1540413897 shutdown_connections 2024-05-03T18:37:52.484 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:51.270+0000 7fdfcb361700 1 --2- 172.21.15.32:0/1540413897 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fdfb003e4c0 0x7fdfb0040980 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:52.484 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:51.270+0000 7fdfcb361700 1 --2- 172.21.15.32:0/1540413897 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdfc4108fa0 0x7fdfc407eb80 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:52.484 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:51.270+0000 7fdfcb361700 1 -- 172.21.15.32:0/1540413897 >> 172.21.15.32:0/1540413897 conn(0x7fdfc4104580 msgr2=0x7fdfc4105130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:37:52.484 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:51.270+0000 7fdfcb361700 1 -- 172.21.15.32:0/1540413897 shutdown_connections 2024-05-03T18:37:52.484 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:51.270+0000 7fdfcb361700 1 -- 172.21.15.32:0/1540413897 wait complete. 2024-05-03T18:37:52.490 INFO:teuthology.orchestra.run.smithi032.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-03T18:37:52.490 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stdout disabled 2024-05-03T18:37:52.499 INFO:teuthology.orchestra.run.smithi032.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-03T18:37:52.499 INFO:teuthology.orchestra.run.smithi032.stdout:systemctl: stdout inactive 2024-05-03T18:37:52.499 INFO:teuthology.orchestra.run.smithi032.stdout:firewalld.service is not enabled 2024-05-03T18:37:52.499 INFO:teuthology.orchestra.run.smithi032.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-05-03T18:37:52.500 INFO:teuthology.orchestra.run.smithi032.stdout:Ceph Dashboard is now available at: 2024-05-03T18:37:52.500 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-03T18:37:52.500 INFO:teuthology.orchestra.run.smithi032.stdout: URL: https://smithi032:8443/ 2024-05-03T18:37:52.501 INFO:teuthology.orchestra.run.smithi032.stdout: User: admin 2024-05-03T18:37:52.501 INFO:teuthology.orchestra.run.smithi032.stdout: Password: 1cg0si4f1j 2024-05-03T18:37:52.501 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-03T18:37:52.501 INFO:teuthology.orchestra.run.smithi032.stdout:Saving cluster configuration to /var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/config directory 2024-05-03T18:37:52.501 INFO:teuthology.orchestra.run.smithi032.stdout:Enabling autotune for osd_memory_target 2024-05-03T18:37:53.711 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:37:53 smithi032 ceph-mon[29763]: mgrmap e13: x(active, since 6s) 2024-05-03T18:37:53.719 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:52.940+0000 7f0fd37df700 1 Processor -- start 2024-05-03T18:37:53.719 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:52.940+0000 7f0fd37df700 1 -- start start 2024-05-03T18:37:53.719 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:52.940+0000 7f0fd37df700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0fcc10a660 0x7f0fcc10aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:53.719 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:52.940+0000 7f0fd37df700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f0fcc108c40 con 0x7f0fcc10a660 2024-05-03T18:37:53.719 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:52.941+0000 7f0fd157b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0fcc10a660 0x7f0fcc10aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:53.719 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:52.941+0000 7f0fd157b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0fcc10a660 0x7f0fcc10aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58364/0 (socket says 172.21.15.32:58364) 2024-05-03T18:37:53.719 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:52.941+0000 7f0fd157b700 1 -- 172.21.15.32:0/1576182953 learned_addr learned my addr 172.21.15.32:0/1576182953 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:37:53.719 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:52.941+0000 7f0fd157b700 1 -- 172.21.15.32:0/1576182953 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fcc108dc0 con 0x7f0fcc10a660 2024-05-03T18:37:53.719 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:52.941+0000 7f0fd157b700 1 --2- 172.21.15.32:0/1576182953 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0fcc10a660 0x7f0fcc10aa80 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f0fc80098a0 tx=0x7f0fc8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f11b88cec91de0e server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:37:53.719 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:52.941+0000 7f0fc3fff700 1 -- 172.21.15.32:0/1576182953 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0fc8004160 con 0x7f0fcc10a660 2024-05-03T18:37:53.719 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:52.941+0000 7f0fc3fff700 1 -- 172.21.15.32:0/1576182953 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0fc8004300 con 0x7f0fcc10a660 2024-05-03T18:37:53.720 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:52.941+0000 7f0fc3fff700 1 -- 172.21.15.32:0/1576182953 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0fc8011a20 con 0x7f0fcc10a660 2024-05-03T18:37:53.720 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:52.942+0000 7f0fd37df700 1 -- 172.21.15.32:0/1576182953 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0fcc10a660 msgr2=0x7f0fcc10aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:37:53.720 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:52.942+0000 7f0fd37df700 1 --2- 172.21.15.32:0/1576182953 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0fcc10a660 0x7f0fcc10aa80 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f0fc80098a0 tx=0x7f0fc8009bb0 comp rx=0 tx=0).stop 2024-05-03T18:37:53.720 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:52.942+0000 7f0fd37df700 1 -- 172.21.15.32:0/1576182953 shutdown_connections 2024-05-03T18:37:53.720 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:52.942+0000 7f0fd37df700 1 --2- 172.21.15.32:0/1576182953 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0fcc10a660 0x7f0fcc10aa80 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:53.720 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:52.942+0000 7f0fd37df700 1 -- 172.21.15.32:0/1576182953 >> 172.21.15.32:0/1576182953 conn(0x7f0fcc104640 msgr2=0x7f0fcc106a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:37:53.720 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:52.942+0000 7f0fd37df700 1 -- 172.21.15.32:0/1576182953 shutdown_connections 2024-05-03T18:37:53.720 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:52.942+0000 7f0fd37df700 1 -- 172.21.15.32:0/1576182953 wait complete. 2024-05-03T18:37:53.720 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:52.943+0000 7f0fd37df700 1 Processor -- start 2024-05-03T18:37:53.720 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:52.943+0000 7f0fd37df700 1 -- start start 2024-05-03T18:37:53.720 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:52.943+0000 7f0fd37df700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0fcc10a660 0x7f0fcc19f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:53.720 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:52.943+0000 7f0fd37df700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f0fcc10d530 con 0x7f0fcc10a660 2024-05-03T18:37:53.720 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:52.943+0000 7f0fd157b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0fcc10a660 0x7f0fcc19f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:53.720 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:52.943+0000 7f0fd157b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0fcc10a660 0x7f0fcc19f1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58380/0 (socket says 172.21.15.32:58380) 2024-05-03T18:37:53.720 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:52.943+0000 7f0fd157b700 1 -- 172.21.15.32:0/1515702729 learned_addr learned my addr 172.21.15.32:0/1515702729 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:37:53.720 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:52.943+0000 7f0fd157b700 1 -- 172.21.15.32:0/1515702729 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fcc19f720 con 0x7f0fcc10a660 2024-05-03T18:37:53.721 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:52.943+0000 7f0fd157b700 1 --2- 172.21.15.32:0/1515702729 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0fcc10a660 0x7f0fcc19f1e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f0fc80119f0 tx=0x7f0fc8011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:37:53.721 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:52.944+0000 7f0fc27fc700 1 -- 172.21.15.32:0/1515702729 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0fc80039e0 con 0x7f0fcc10a660 2024-05-03T18:37:53.721 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:52.944+0000 7f0fc27fc700 1 -- 172.21.15.32:0/1515702729 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0fc801b8f0 con 0x7f0fcc10a660 2024-05-03T18:37:53.721 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:52.944+0000 7f0fd37df700 1 -- 172.21.15.32:0/1515702729 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fcc19f9b0 con 0x7f0fcc10a660 2024-05-03T18:37:53.721 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:52.944+0000 7f0fc27fc700 1 -- 172.21.15.32:0/1515702729 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0fc80237a0 con 0x7f0fcc10a660 2024-05-03T18:37:53.721 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:52.944+0000 7f0fd37df700 1 -- 172.21.15.32:0/1515702729 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fcc1a26a0 con 0x7f0fcc10a660 2024-05-03T18:37:53.721 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:52.945+0000 7f0fc27fc700 1 -- 172.21.15.32:0/1515702729 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f0fc802c3f0 con 0x7f0fcc10a660 2024-05-03T18:37:53.721 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:52.945+0000 7f0fd37df700 1 -- 172.21.15.32:0/1515702729 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0fcc10d6b0 con 0x7f0fcc10a660 2024-05-03T18:37:53.721 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:52.945+0000 7f0fc27fc700 1 --2- 172.21.15.32:0/1515702729 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f0fb803e570 0x7f0fb8040a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:53.721 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:52.945+0000 7f0fc27fc700 1 -- 172.21.15.32:0/1515702729 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f0fc8054170 con 0x7f0fcc10a660 2024-05-03T18:37:53.721 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:52.945+0000 7f0fd0d7a700 1 --2- 172.21.15.32:0/1515702729 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f0fb803e570 0x7f0fb8040a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:53.721 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:52.945+0000 7f0fd0d7a700 1 --2- 172.21.15.32:0/1515702729 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f0fb803e570 0x7f0fb8040a30 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f0fbc006fd0 tx=0x7f0fbc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:37:53.721 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:52.950+0000 7f0fc27fc700 1 -- 172.21.15.32:0/1515702729 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f0fc801f030 con 0x7f0fcc10a660 2024-05-03T18:37:53.721 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:53.163+0000 7f0fd37df700 1 -- 172.21.15.32:0/1515702729 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7f0fcc064b00 con 0x7f0fcc10a660 2024-05-03T18:37:53.721 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:53.163+0000 7f0fc27fc700 1 -- 172.21.15.32:0/1515702729 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7f0fc80105e0 con 0x7f0fcc10a660 2024-05-03T18:37:53.721 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:53.165+0000 7f0fd37df700 1 -- 172.21.15.32:0/1515702729 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f0fb803e570 msgr2=0x7f0fb8040a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:37:53.722 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:53.165+0000 7f0fd37df700 1 --2- 172.21.15.32:0/1515702729 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f0fb803e570 0x7f0fb8040a30 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f0fbc006fd0 tx=0x7f0fbc006e40 comp rx=0 tx=0).stop 2024-05-03T18:37:53.722 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:53.165+0000 7f0fd37df700 1 -- 172.21.15.32:0/1515702729 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0fcc10a660 msgr2=0x7f0fcc19f1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:37:53.722 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:53.165+0000 7f0fd37df700 1 --2- 172.21.15.32:0/1515702729 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0fcc10a660 0x7f0fcc19f1e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f0fc80119f0 tx=0x7f0fc8011ad0 comp rx=0 tx=0).stop 2024-05-03T18:37:53.722 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:53.165+0000 7f0fd37df700 1 -- 172.21.15.32:0/1515702729 shutdown_connections 2024-05-03T18:37:53.722 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:53.165+0000 7f0fd37df700 1 --2- 172.21.15.32:0/1515702729 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f0fb803e570 0x7f0fb8040a30 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:53.722 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:53.165+0000 7f0fd37df700 1 --2- 172.21.15.32:0/1515702729 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0fcc10a660 0x7f0fcc19f1e0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:53.722 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:53.165+0000 7f0fd37df700 1 -- 172.21.15.32:0/1515702729 >> 172.21.15.32:0/1515702729 conn(0x7f0fcc104640 msgr2=0x7f0fcc105110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:37:53.722 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:53.165+0000 7f0fd37df700 1 -- 172.21.15.32:0/1515702729 shutdown_connections 2024-05-03T18:37:53.722 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:53.165+0000 7f0fd37df700 1 -- 172.21.15.32:0/1515702729 wait complete. 2024-05-03T18:37:54.977 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:54.139+0000 7f0457537700 1 Processor -- start 2024-05-03T18:37:54.977 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:54.140+0000 7f0457537700 1 -- start start 2024-05-03T18:37:54.977 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:54.140+0000 7f0457537700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0450109010 0x7f0450109430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:54.977 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:54.140+0000 7f0457537700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f0450109a00 con 0x7f0450109010 2024-05-03T18:37:54.977 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:54.140+0000 7f04552d3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0450109010 0x7f0450109430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:54.978 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:54.140+0000 7f04552d3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0450109010 0x7f0450109430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45816/0 (socket says 172.21.15.32:45816) 2024-05-03T18:37:54.978 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:54.140+0000 7f04552d3700 1 -- 172.21.15.32:0/289669940 learned_addr learned my addr 172.21.15.32:0/289669940 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:37:54.978 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:54.140+0000 7f04552d3700 1 -- 172.21.15.32:0/289669940 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f045010a170 con 0x7f0450109010 2024-05-03T18:37:54.978 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:54.141+0000 7f04552d3700 1 --2- 172.21.15.32:0/289669940 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0450109010 0x7f0450109430 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f04400098a0 tx=0x7f0440009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18b012acbc7447fc server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:37:54.978 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:54.141+0000 7f0447fff700 1 -- 172.21.15.32:0/289669940 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0440004160 con 0x7f0450109010 2024-05-03T18:37:54.978 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:54.141+0000 7f0447fff700 1 -- 172.21.15.32:0/289669940 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0440004300 con 0x7f0450109010 2024-05-03T18:37:54.978 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:54.141+0000 7f0447fff700 1 -- 172.21.15.32:0/289669940 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0440011a20 con 0x7f0450109010 2024-05-03T18:37:54.978 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:54.141+0000 7f0457537700 1 -- 172.21.15.32:0/289669940 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0450109010 msgr2=0x7f0450109430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:37:54.978 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:54.141+0000 7f0457537700 1 --2- 172.21.15.32:0/289669940 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0450109010 0x7f0450109430 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f04400098a0 tx=0x7f0440009bb0 comp rx=0 tx=0).stop 2024-05-03T18:37:54.978 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:54.141+0000 7f0457537700 1 -- 172.21.15.32:0/289669940 shutdown_connections 2024-05-03T18:37:54.978 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:54.141+0000 7f0457537700 1 --2- 172.21.15.32:0/289669940 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0450109010 0x7f0450109430 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:54.978 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:54.141+0000 7f0457537700 1 -- 172.21.15.32:0/289669940 >> 172.21.15.32:0/289669940 conn(0x7f0450104650 msgr2=0x7f0450106a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:37:54.978 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:54.142+0000 7f0457537700 1 -- 172.21.15.32:0/289669940 shutdown_connections 2024-05-03T18:37:54.979 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:54.142+0000 7f0457537700 1 -- 172.21.15.32:0/289669940 wait complete. 2024-05-03T18:37:54.979 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:54.142+0000 7f0457537700 1 Processor -- start 2024-05-03T18:37:54.979 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:54.142+0000 7f0457537700 1 -- start start 2024-05-03T18:37:54.979 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:54.142+0000 7f0457537700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0450109010 0x7f045019cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:54.979 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:54.142+0000 7f0457537700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f045010d520 con 0x7f0450109010 2024-05-03T18:37:54.979 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:54.143+0000 7f04552d3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0450109010 0x7f045019cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:54.979 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:54.143+0000 7f04552d3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0450109010 0x7f045019cea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45824/0 (socket says 172.21.15.32:45824) 2024-05-03T18:37:54.979 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:54.143+0000 7f04552d3700 1 -- 172.21.15.32:0/2264905238 learned_addr learned my addr 172.21.15.32:0/2264905238 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:37:54.979 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:54.143+0000 7f04552d3700 1 -- 172.21.15.32:0/2264905238 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f045019d3e0 con 0x7f0450109010 2024-05-03T18:37:54.979 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:54.143+0000 7f04552d3700 1 --2- 172.21.15.32:0/2264905238 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0450109010 0x7f045019cea0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f0440000c00 tx=0x7f0440011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:37:54.979 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:54.143+0000 7f04467fc700 1 -- 172.21.15.32:0/2264905238 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0440003aa0 con 0x7f0450109010 2024-05-03T18:37:54.979 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:54.143+0000 7f04467fc700 1 -- 172.21.15.32:0/2264905238 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f044001b8f0 con 0x7f0450109010 2024-05-03T18:37:54.979 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:54.143+0000 7f04467fc700 1 -- 172.21.15.32:0/2264905238 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0440023890 con 0x7f0450109010 2024-05-03T18:37:54.979 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:54.143+0000 7f0457537700 1 -- 172.21.15.32:0/2264905238 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f045019d670 con 0x7f0450109010 2024-05-03T18:37:54.980 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:54.143+0000 7f0457537700 1 -- 172.21.15.32:0/2264905238 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04501a0360 con 0x7f0450109010 2024-05-03T18:37:54.980 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:54.144+0000 7f04467fc700 1 -- 172.21.15.32:0/2264905238 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f044001b440 con 0x7f0450109010 2024-05-03T18:37:54.980 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:54.144+0000 7f0457537700 1 -- 172.21.15.32:0/2264905238 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0450064b00 con 0x7f0450109010 2024-05-03T18:37:54.980 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:54.144+0000 7f04467fc700 1 --2- 172.21.15.32:0/2264905238 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f043c03e560 0x7f043c040a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:54.980 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:54.144+0000 7f04467fc700 1 -- 172.21.15.32:0/2264905238 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f0440054020 con 0x7f0450109010 2024-05-03T18:37:54.980 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:54.145+0000 7f0454ad2700 1 --2- 172.21.15.32:0/2264905238 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f043c03e560 0x7f043c040a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:54.980 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:54.145+0000 7f0454ad2700 1 --2- 172.21.15.32:0/2264905238 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f043c03e560 0x7f043c040a20 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f044c006fd0 tx=0x7f044c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:37:54.980 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:54.150+0000 7f04467fc700 1 -- 172.21.15.32:0/2264905238 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f044001f030 con 0x7f0450109010 2024-05-03T18:37:54.980 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:54.483+0000 7f0457537700 1 -- 172.21.15.32:0/2264905238 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7f045019db50 con 0x7f0450109010 2024-05-03T18:37:54.980 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:54.485+0000 7f04467fc700 1 -- 172.21.15.32:0/2264905238 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v27)=0 set mgr/dashboard/cluster/status v27) ==== 153+0+0 (secure 0 0 0) 0x7f045019db50 con 0x7f0450109010 2024-05-03T18:37:54.980 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:54.487+0000 7f0457537700 1 -- 172.21.15.32:0/2264905238 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f043c03e560 msgr2=0x7f043c040a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:37:54.980 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:54.487+0000 7f0457537700 1 --2- 172.21.15.32:0/2264905238 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f043c03e560 0x7f043c040a20 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f044c006fd0 tx=0x7f044c006e40 comp rx=0 tx=0).stop 2024-05-03T18:37:54.980 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:54.487+0000 7f0457537700 1 -- 172.21.15.32:0/2264905238 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0450109010 msgr2=0x7f045019cea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:37:54.980 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:54.487+0000 7f0457537700 1 --2- 172.21.15.32:0/2264905238 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0450109010 0x7f045019cea0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f0440000c00 tx=0x7f0440011aa0 comp rx=0 tx=0).stop 2024-05-03T18:37:54.981 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:54.487+0000 7f0457537700 1 -- 172.21.15.32:0/2264905238 shutdown_connections 2024-05-03T18:37:54.981 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:54.487+0000 7f0457537700 1 --2- 172.21.15.32:0/2264905238 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f043c03e560 0x7f043c040a20 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:54.981 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:54.488+0000 7f0457537700 1 --2- 172.21.15.32:0/2264905238 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0450109010 0x7f045019cea0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:54.981 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:54.488+0000 7f0457537700 1 -- 172.21.15.32:0/2264905238 >> 172.21.15.32:0/2264905238 conn(0x7f0450104650 msgr2=0x7f0450105150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:37:54.981 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:54.488+0000 7f0457537700 1 -- 172.21.15.32:0/2264905238 shutdown_connections 2024-05-03T18:37:54.981 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr 2024-05-03T18:37:54.488+0000 7f0457537700 1 -- 172.21.15.32:0/2264905238 wait complete. 2024-05-03T18:37:54.981 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-05-03T18:37:54.981 INFO:teuthology.orchestra.run.smithi032.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-05-03T18:37:54.981 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-03T18:37:54.981 INFO:teuthology.orchestra.run.smithi032.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid f500455e-097b-11ef-bc97-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-05-03T18:37:54.981 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-03T18:37:54.981 INFO:teuthology.orchestra.run.smithi032.stdout:Or, if you are only running a single cluster on this host: 2024-05-03T18:37:54.981 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-03T18:37:54.981 INFO:teuthology.orchestra.run.smithi032.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-05-03T18:37:54.981 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-03T18:37:54.981 INFO:teuthology.orchestra.run.smithi032.stdout:Please consider enabling telemetry to help improve Ceph: 2024-05-03T18:37:54.982 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-03T18:37:54.982 INFO:teuthology.orchestra.run.smithi032.stdout: ceph telemetry on 2024-05-03T18:37:54.982 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-03T18:37:54.982 INFO:teuthology.orchestra.run.smithi032.stdout:For more information see: 2024-05-03T18:37:54.982 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-03T18:37:54.982 INFO:teuthology.orchestra.run.smithi032.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-05-03T18:37:54.982 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-03T18:37:54.982 INFO:teuthology.orchestra.run.smithi032.stdout:Bootstrap complete. 2024-05-03T18:37:55.021 INFO:tasks.cephadm:Fetching config... 2024-05-03T18:37:55.021 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-05-03T18:37:55.021 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-05-03T18:37:55.038 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-05-03T18:37:55.038 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-05-03T18:37:55.038 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-05-03T18:37:55.094 INFO:tasks.cephadm:Fetching mon keyring... 2024-05-03T18:37:55.094 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-05-03T18:37:55.095 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/mon.a/keyring of=/dev/stdout 2024-05-03T18:37:55.159 INFO:tasks.cephadm:Fetching pub ssh key... 2024-05-03T18:37:55.159 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-05-03T18:37:55.160 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-05-03T18:37:55.216 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-05-03T18:37:55.217 DEBUG:teuthology.orchestra.run.smithi032:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC+oFN0k/X5PxHqlAC26+EPkxhb41cOmEL7M6meXPdUq98nXACJJSdiLyb4jF1uj4IiucK6o6m+0+HF+RctoQfRue4dqzM+Aqc4aoomOevP9qoeBukZkanqw1ja/shNVJoOnQUP9P0XYCI0Y4GQ2VGuvmVN8KGhocbiNaweG3kfhSj+YABVB47Mafot3iv+lMt0pd1hgZZfgb4DbC8T5tvu1rVurzBWrr0NU5de4C0ja2a7J+d7ds+9PRa4BmCZ++4ZwBNXn6G0kgl6jM2cvCuURY8ulL1tmSdJP1WRXRV0HIIQbo6CL0R2V+3Idfu5h6VYvyVN/i/ZaXu0IkkrSKtUO5cr6UC9yIn2096yV/8w5KoU7J0Q29fe9J8le4VBHHbP/w55/1sdrPht8RpyA4ER+zQyX1BknR578EE5knbuEd/s790NA6pm/edy2gTNz2b56uTbOTVQlt1ViEZL96c31MAHYX71AUQVQrd2pw8DAxpZYaipivTvFLMTOTKgORE= ceph-f500455e-097b-11ef-bc97-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-03T18:37:55.292 INFO:teuthology.orchestra.run.smithi032.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC+oFN0k/X5PxHqlAC26+EPkxhb41cOmEL7M6meXPdUq98nXACJJSdiLyb4jF1uj4IiucK6o6m+0+HF+RctoQfRue4dqzM+Aqc4aoomOevP9qoeBukZkanqw1ja/shNVJoOnQUP9P0XYCI0Y4GQ2VGuvmVN8KGhocbiNaweG3kfhSj+YABVB47Mafot3iv+lMt0pd1hgZZfgb4DbC8T5tvu1rVurzBWrr0NU5de4C0ja2a7J+d7ds+9PRa4BmCZ++4ZwBNXn6G0kgl6jM2cvCuURY8ulL1tmSdJP1WRXRV0HIIQbo6CL0R2V+3Idfu5h6VYvyVN/i/ZaXu0IkkrSKtUO5cr6UC9yIn2096yV/8w5KoU7J0Q29fe9J8le4VBHHbP/w55/1sdrPht8RpyA4ER+zQyX1BknR578EE5knbuEd/s790NA6pm/edy2gTNz2b56uTbOTVQlt1ViEZL96c31MAHYX71AUQVQrd2pw8DAxpZYaipivTvFLMTOTKgORE= ceph-f500455e-097b-11ef-bc97-c7b262605968 2024-05-03T18:37:55.304 DEBUG:teuthology.orchestra.run.smithi045:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC+oFN0k/X5PxHqlAC26+EPkxhb41cOmEL7M6meXPdUq98nXACJJSdiLyb4jF1uj4IiucK6o6m+0+HF+RctoQfRue4dqzM+Aqc4aoomOevP9qoeBukZkanqw1ja/shNVJoOnQUP9P0XYCI0Y4GQ2VGuvmVN8KGhocbiNaweG3kfhSj+YABVB47Mafot3iv+lMt0pd1hgZZfgb4DbC8T5tvu1rVurzBWrr0NU5de4C0ja2a7J+d7ds+9PRa4BmCZ++4ZwBNXn6G0kgl6jM2cvCuURY8ulL1tmSdJP1WRXRV0HIIQbo6CL0R2V+3Idfu5h6VYvyVN/i/ZaXu0IkkrSKtUO5cr6UC9yIn2096yV/8w5KoU7J0Q29fe9J8le4VBHHbP/w55/1sdrPht8RpyA4ER+zQyX1BknR578EE5knbuEd/s790NA6pm/edy2gTNz2b56uTbOTVQlt1ViEZL96c31MAHYX71AUQVQrd2pw8DAxpZYaipivTvFLMTOTKgORE= ceph-f500455e-097b-11ef-bc97-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-03T18:37:55.346 INFO:teuthology.orchestra.run.smithi045.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC+oFN0k/X5PxHqlAC26+EPkxhb41cOmEL7M6meXPdUq98nXACJJSdiLyb4jF1uj4IiucK6o6m+0+HF+RctoQfRue4dqzM+Aqc4aoomOevP9qoeBukZkanqw1ja/shNVJoOnQUP9P0XYCI0Y4GQ2VGuvmVN8KGhocbiNaweG3kfhSj+YABVB47Mafot3iv+lMt0pd1hgZZfgb4DbC8T5tvu1rVurzBWrr0NU5de4C0ja2a7J+d7ds+9PRa4BmCZ++4ZwBNXn6G0kgl6jM2cvCuURY8ulL1tmSdJP1WRXRV0HIIQbo6CL0R2V+3Idfu5h6VYvyVN/i/ZaXu0IkkrSKtUO5cr6UC9yIn2096yV/8w5KoU7J0Q29fe9J8le4VBHHbP/w55/1sdrPht8RpyA4ER+zQyX1BknR578EE5knbuEd/s790NA6pm/edy2gTNz2b56uTbOTVQlt1ViEZL96c31MAHYX71AUQVQrd2pw8DAxpZYaipivTvFLMTOTKgORE= ceph-f500455e-097b-11ef-bc97-c7b262605968 2024-05-03T18:37:55.358 DEBUG:teuthology.orchestra.run.smithi113:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC+oFN0k/X5PxHqlAC26+EPkxhb41cOmEL7M6meXPdUq98nXACJJSdiLyb4jF1uj4IiucK6o6m+0+HF+RctoQfRue4dqzM+Aqc4aoomOevP9qoeBukZkanqw1ja/shNVJoOnQUP9P0XYCI0Y4GQ2VGuvmVN8KGhocbiNaweG3kfhSj+YABVB47Mafot3iv+lMt0pd1hgZZfgb4DbC8T5tvu1rVurzBWrr0NU5de4C0ja2a7J+d7ds+9PRa4BmCZ++4ZwBNXn6G0kgl6jM2cvCuURY8ulL1tmSdJP1WRXRV0HIIQbo6CL0R2V+3Idfu5h6VYvyVN/i/ZaXu0IkkrSKtUO5cr6UC9yIn2096yV/8w5KoU7J0Q29fe9J8le4VBHHbP/w55/1sdrPht8RpyA4ER+zQyX1BknR578EE5knbuEd/s790NA6pm/edy2gTNz2b56uTbOTVQlt1ViEZL96c31MAHYX71AUQVQrd2pw8DAxpZYaipivTvFLMTOTKgORE= ceph-f500455e-097b-11ef-bc97-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-03T18:37:55.397 INFO:teuthology.orchestra.run.smithi113.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC+oFN0k/X5PxHqlAC26+EPkxhb41cOmEL7M6meXPdUq98nXACJJSdiLyb4jF1uj4IiucK6o6m+0+HF+RctoQfRue4dqzM+Aqc4aoomOevP9qoeBukZkanqw1ja/shNVJoOnQUP9P0XYCI0Y4GQ2VGuvmVN8KGhocbiNaweG3kfhSj+YABVB47Mafot3iv+lMt0pd1hgZZfgb4DbC8T5tvu1rVurzBWrr0NU5de4C0ja2a7J+d7ds+9PRa4BmCZ++4ZwBNXn6G0kgl6jM2cvCuURY8ulL1tmSdJP1WRXRV0HIIQbo6CL0R2V+3Idfu5h6VYvyVN/i/ZaXu0IkkrSKtUO5cr6UC9yIn2096yV/8w5KoU7J0Q29fe9J8le4VBHHbP/w55/1sdrPht8RpyA4ER+zQyX1BknR578EE5knbuEd/s790NA6pm/edy2gTNz2b56uTbOTVQlt1ViEZL96c31MAHYX71AUQVQrd2pw8DAxpZYaipivTvFLMTOTKgORE= ceph-f500455e-097b-11ef-bc97-c7b262605968 2024-05-03T18:37:55.409 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f500455e-097b-11ef-bc97-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-05-03T18:37:55.567 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:37:55 smithi032 ceph-mon[29763]: from='client.? 172.21.15.32:0/2264905238' entity='client.admin' 2024-05-03T18:37:55.655 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:37:57.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:37:57.558+0000 7f69b22b5700 1 -- 172.21.15.32:0/2979251862 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f69ac1039e0 msgr2=0x7f69ac105dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:37:57.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:37:57.558+0000 7f69b22b5700 1 --2- 172.21.15.32:0/2979251862 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f69ac1039e0 0x7f69ac105dd0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f69a0009910 tx=0x7f69a0009c20 comp rx=0 tx=0).stop 2024-05-03T18:37:57.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:37:57.558+0000 7f69b22b5700 1 -- 172.21.15.32:0/2979251862 shutdown_connections 2024-05-03T18:37:57.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:37:57.558+0000 7f69b22b5700 1 --2- 172.21.15.32:0/2979251862 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f69ac1039e0 0x7f69ac105dd0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:57.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:37:57.558+0000 7f69b22b5700 1 -- 172.21.15.32:0/2979251862 >> 172.21.15.32:0/2979251862 conn(0x7f69ac0fd3b0 msgr2=0x7f69ac0ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:37:57.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:37:57.558+0000 7f69b22b5700 1 -- 172.21.15.32:0/2979251862 shutdown_connections 2024-05-03T18:37:57.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:37:57.558+0000 7f69b22b5700 1 -- 172.21.15.32:0/2979251862 wait complete. 2024-05-03T18:37:57.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:37:57.558+0000 7f69b22b5700 1 Processor -- start 2024-05-03T18:37:57.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:37:57.558+0000 7f69b22b5700 1 -- start start 2024-05-03T18:37:57.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:37:57.559+0000 7f69b22b5700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f69ac1039e0 0x7f69ac198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:57.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:37:57.559+0000 7f69b22b5700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f69ac108e30 con 0x7f69ac1039e0 2024-05-03T18:37:57.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:37:57.559+0000 7f69abfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f69ac1039e0 0x7f69ac198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:57.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:37:57.559+0000 7f69abfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f69ac1039e0 0x7f69ac198500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45832/0 (socket says 172.21.15.32:45832) 2024-05-03T18:37:57.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:37:57.559+0000 7f69abfff700 1 -- 172.21.15.32:0/3811792347 learned_addr learned my addr 172.21.15.32:0/3811792347 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:37:57.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:37:57.559+0000 7f69abfff700 1 -- 172.21.15.32:0/3811792347 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69ac198a40 con 0x7f69ac1039e0 2024-05-03T18:37:57.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:37:57.559+0000 7f69abfff700 1 --2- 172.21.15.32:0/3811792347 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f69ac1039e0 0x7f69ac198500 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f69a0005b40 tx=0x7f69a0004f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:37:57.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:37:57.559+0000 7f69a97fa700 1 -- 172.21.15.32:0/3811792347 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f69a0027070 con 0x7f69ac1039e0 2024-05-03T18:37:57.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:37:57.559+0000 7f69b22b5700 1 -- 172.21.15.32:0/3811792347 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69ac198d30 con 0x7f69ac1039e0 2024-05-03T18:37:57.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:37:57.559+0000 7f69a97fa700 1 -- 172.21.15.32:0/3811792347 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f69a000be10 con 0x7f69ac1039e0 2024-05-03T18:37:57.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:37:57.559+0000 7f69a97fa700 1 -- 172.21.15.32:0/3811792347 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f69a0022de0 con 0x7f69ac1039e0 2024-05-03T18:37:57.561 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:37:57.560+0000 7f69b22b5700 1 -- 172.21.15.32:0/3811792347 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69ac19b970 con 0x7f69ac1039e0 2024-05-03T18:37:57.562 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:37:57.560+0000 7f69a97fa700 1 -- 172.21.15.32:0/3811792347 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f69a002a070 con 0x7f69ac1039e0 2024-05-03T18:37:57.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:37:57.560+0000 7f69b22b5700 1 -- 172.21.15.32:0/3811792347 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69ac1095b0 con 0x7f69ac1039e0 2024-05-03T18:37:57.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:37:57.561+0000 7f69a97fa700 1 --2- 172.21.15.32:0/3811792347 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f699403e510 0x7f69940409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:37:57.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:37:57.561+0000 7f69a97fa700 1 -- 172.21.15.32:0/3811792347 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f69a001e030 con 0x7f69ac1039e0 2024-05-03T18:37:57.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:37:57.561+0000 7f69ab7fe700 1 --2- 172.21.15.32:0/3811792347 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f699403e510 0x7f69940409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:37:57.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:37:57.561+0000 7f69ab7fe700 1 --2- 172.21.15.32:0/3811792347 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f699403e510 0x7f69940409d0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f699c006fd0 tx=0x7f699c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:37:57.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:37:57.566+0000 7f69a97fa700 1 -- 172.21.15.32:0/3811792347 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f69a0013070 con 0x7f69ac1039e0 2024-05-03T18:37:57.777 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:37:57.777+0000 7f69b22b5700 1 -- 172.21.15.32:0/3811792347 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f69ac02e700 con 0x7f69ac1039e0 2024-05-03T18:37:57.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:37:57.779+0000 7f69a97fa700 1 -- 172.21.15.32:0/3811792347 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f69a0031070 con 0x7f69ac1039e0 2024-05-03T18:37:57.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:37:57.781+0000 7f69b22b5700 1 -- 172.21.15.32:0/3811792347 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f699403e510 msgr2=0x7f69940409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:37:57.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:37:57.781+0000 7f69b22b5700 1 --2- 172.21.15.32:0/3811792347 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f699403e510 0x7f69940409d0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f699c006fd0 tx=0x7f699c006e40 comp rx=0 tx=0).stop 2024-05-03T18:37:57.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:37:57.781+0000 7f69b22b5700 1 -- 172.21.15.32:0/3811792347 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f69ac1039e0 msgr2=0x7f69ac198500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:37:57.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:37:57.781+0000 7f69b22b5700 1 --2- 172.21.15.32:0/3811792347 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f69ac1039e0 0x7f69ac198500 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f69a0005b40 tx=0x7f69a0004f80 comp rx=0 tx=0).stop 2024-05-03T18:37:57.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:37:57.782+0000 7f69b22b5700 1 -- 172.21.15.32:0/3811792347 shutdown_connections 2024-05-03T18:37:57.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:37:57.782+0000 7f69b22b5700 1 --2- 172.21.15.32:0/3811792347 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f699403e510 0x7f69940409d0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:57.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:37:57.782+0000 7f69b22b5700 1 --2- 172.21.15.32:0/3811792347 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f69ac1039e0 0x7f69ac198500 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:37:57.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:37:57.782+0000 7f69b22b5700 1 -- 172.21.15.32:0/3811792347 >> 172.21.15.32:0/3811792347 conn(0x7f69ac0fd3b0 msgr2=0x7f69ac0fdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:37:57.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:37:57.782+0000 7f69b22b5700 1 -- 172.21.15.32:0/3811792347 shutdown_connections 2024-05-03T18:37:57.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:37:57.782+0000 7f69b22b5700 1 -- 172.21.15.32:0/3811792347 wait complete. 2024-05-03T18:37:58.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:37:57 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:37:58.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:37:57 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:37:58.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:37:57 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi032", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:37:58.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:37:57 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:37:58.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:37:57 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:37:58.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:37:57 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:37:58.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:37:57 smithi032 ceph-mon[29763]: from='client.? 172.21.15.32:0/3811792347' entity='client.admin' 2024-05-03T18:37:58.378 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-05-03T18:37:58.378 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f500455e-097b-11ef-bc97-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-05-03T18:37:58.621 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:37:59.034 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:37:58 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:37:59.034 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:37:58 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:00.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:00.003+0000 7f65265d6700 1 -- 172.21.15.32:0/2142170407 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6520104570 msgr2=0x7f6520104990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:38:00.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:00.003+0000 7f65265d6700 1 --2- 172.21.15.32:0/2142170407 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6520104570 0x7f6520104990 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6510009910 tx=0x7f6510009c20 comp rx=0 tx=0).stop 2024-05-03T18:38:00.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:00.003+0000 7f65265d6700 1 -- 172.21.15.32:0/2142170407 shutdown_connections 2024-05-03T18:38:00.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:00.003+0000 7f65265d6700 1 --2- 172.21.15.32:0/2142170407 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6520104570 0x7f6520104990 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:00.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:00.003+0000 7f65265d6700 1 -- 172.21.15.32:0/2142170407 >> 172.21.15.32:0/2142170407 conn(0x7f65200fe4f0 msgr2=0x7f6520100910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:38:00.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:00.003+0000 7f65265d6700 1 -- 172.21.15.32:0/2142170407 shutdown_connections 2024-05-03T18:38:00.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:00.003+0000 7f65265d6700 1 -- 172.21.15.32:0/2142170407 wait complete. 2024-05-03T18:38:00.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:00.003+0000 7f65265d6700 1 Processor -- start 2024-05-03T18:38:00.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:00.003+0000 7f65265d6700 1 -- start start 2024-05-03T18:38:00.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:00.004+0000 7f65265d6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6520104570 0x7f6520077160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:38:00.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:00.004+0000 7f65265d6700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f6520108ef0 con 0x7f6520104570 2024-05-03T18:38:00.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:00.004+0000 7f651ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6520104570 0x7f6520077160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:38:00.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:00.004+0000 7f651ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6520104570 0x7f6520077160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45862/0 (socket says 172.21.15.32:45862) 2024-05-03T18:38:00.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:00.004+0000 7f651ffff700 1 -- 172.21.15.32:0/2242549598 learned_addr learned my addr 172.21.15.32:0/2242549598 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:38:00.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:00.004+0000 7f651ffff700 1 -- 172.21.15.32:0/2242549598 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65200776a0 con 0x7f6520104570 2024-05-03T18:38:00.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:00.004+0000 7f651ffff700 1 --2- 172.21.15.32:0/2242549598 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6520104570 0x7f6520077160 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6510005b40 tx=0x7f6510004f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:38:00.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:00.004+0000 7f651d7fa700 1 -- 172.21.15.32:0/2242549598 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6510027070 con 0x7f6520104570 2024-05-03T18:38:00.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:00.004+0000 7f651d7fa700 1 -- 172.21.15.32:0/2242549598 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f651000be10 con 0x7f6520104570 2024-05-03T18:38:00.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:00.004+0000 7f65265d6700 1 -- 172.21.15.32:0/2242549598 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6520075890 con 0x7f6520104570 2024-05-03T18:38:00.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:00.004+0000 7f651d7fa700 1 -- 172.21.15.32:0/2242549598 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6510022de0 con 0x7f6520104570 2024-05-03T18:38:00.007 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:00.004+0000 7f65265d6700 1 -- 172.21.15.32:0/2242549598 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6520075dd0 con 0x7f6520104570 2024-05-03T18:38:00.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:00.006+0000 7f651d7fa700 1 -- 172.21.15.32:0/2242549598 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f651002a070 con 0x7f6520104570 2024-05-03T18:38:00.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:00.006+0000 7f65265d6700 1 -- 172.21.15.32:0/2242549598 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6520109670 con 0x7f6520104570 2024-05-03T18:38:00.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:00.006+0000 7f651d7fa700 1 --2- 172.21.15.32:0/2242549598 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f650803e570 0x7f6508040a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:38:00.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:00.006+0000 7f651d7fa700 1 -- 172.21.15.32:0/2242549598 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f651001e030 con 0x7f6520104570 2024-05-03T18:38:00.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:00.006+0000 7f651f7fe700 1 --2- 172.21.15.32:0/2242549598 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f650803e570 0x7f6508040a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:38:00.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:00.006+0000 7f651f7fe700 1 --2- 172.21.15.32:0/2242549598 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f650803e570 0x7f6508040a30 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f6514006fd0 tx=0x7f6514006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:38:00.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:00.011+0000 7f651d7fa700 1 -- 172.21.15.32:0/2242549598 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f6510013070 con 0x7f6520104570 2024-05-03T18:38:00.230 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:00.229+0000 7f65265d6700 1 -- 172.21.15.32:0/2242549598 --> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7f6520191860 con 0x7f650803e570 2024-05-03T18:38:00.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:00.232+0000 7f651d7fa700 1 -- 172.21.15.32:0/2242549598 <== mgr.14152 v2:172.21.15.32:6800/1796547904 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f6520191860 con 0x7f650803e570 2024-05-03T18:38:00.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:00.235+0000 7f65265d6700 1 -- 172.21.15.32:0/2242549598 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f650803e570 msgr2=0x7f6508040a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:38:00.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:00.235+0000 7f65265d6700 1 --2- 172.21.15.32:0/2242549598 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f650803e570 0x7f6508040a30 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f6514006fd0 tx=0x7f6514006e40 comp rx=0 tx=0).stop 2024-05-03T18:38:00.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:00.235+0000 7f65265d6700 1 -- 172.21.15.32:0/2242549598 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6520104570 msgr2=0x7f6520077160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:38:00.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:00.235+0000 7f65265d6700 1 --2- 172.21.15.32:0/2242549598 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6520104570 0x7f6520077160 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6510005b40 tx=0x7f6510004f80 comp rx=0 tx=0).stop 2024-05-03T18:38:00.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:00.235+0000 7f65265d6700 1 -- 172.21.15.32:0/2242549598 shutdown_connections 2024-05-03T18:38:00.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:00.235+0000 7f65265d6700 1 --2- 172.21.15.32:0/2242549598 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f650803e570 0x7f6508040a30 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:00.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:00.235+0000 7f65265d6700 1 --2- 172.21.15.32:0/2242549598 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6520104570 0x7f6520077160 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:00.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:00.235+0000 7f65265d6700 1 -- 172.21.15.32:0/2242549598 >> 172.21.15.32:0/2242549598 conn(0x7f65200fe4f0 msgr2=0x7f65200fefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:38:00.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:00.235+0000 7f65265d6700 1 -- 172.21.15.32:0/2242549598 shutdown_connections 2024-05-03T18:38:00.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:00.235+0000 7f65265d6700 1 -- 172.21.15.32:0/2242549598 wait complete. 2024-05-03T18:38:00.993 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi045 2024-05-03T18:38:00.993 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-05-03T18:38:00.993 DEBUG:teuthology.orchestra.run.smithi045:> dd of=/etc/ceph/ceph.conf 2024-05-03T18:38:01.012 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-05-03T18:38:01.013 DEBUG:teuthology.orchestra.run.smithi045:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-03T18:38:01.071 INFO:tasks.cephadm:Adding host smithi045 to orchestrator... 2024-05-03T18:38:01.072 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f500455e-097b-11ef-bc97-c7b262605968 -- ceph orch host add smithi045 2024-05-03T18:38:01.318 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:38:01.556 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:01 smithi032 ceph-mon[29763]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:38:01.556 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:01 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:01.556 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:01 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:38:01.556 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:01 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:38:01.556 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:01 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:38:01.556 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:01 smithi032 ceph-mon[29763]: Updating smithi032:/etc/ceph/ceph.conf 2024-05-03T18:38:01.556 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:01 smithi032 ceph-mon[29763]: Updating smithi032:/var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-03T18:38:01.556 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:01 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:01.556 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:01 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:01.556 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:01 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:02.495 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:02 smithi032 ceph-mon[29763]: Updating smithi032:/etc/ceph/ceph.client.admin.keyring 2024-05-03T18:38:02.495 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:02 smithi032 ceph-mon[29763]: Updating smithi032:/var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-03T18:38:02.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:02.834+0000 7f053c661700 1 -- 172.21.15.32:0/2509761609 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0534104070 msgr2=0x7f0534104450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:38:02.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:02.834+0000 7f053c661700 1 --2- 172.21.15.32:0/2509761609 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0534104070 0x7f0534104450 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f0530009910 tx=0x7f0530009c20 comp rx=0 tx=0).stop 2024-05-03T18:38:02.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:02.834+0000 7f053c661700 1 -- 172.21.15.32:0/2509761609 shutdown_connections 2024-05-03T18:38:02.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:02.834+0000 7f053c661700 1 --2- 172.21.15.32:0/2509761609 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0534104070 0x7f0534104450 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:02.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:02.834+0000 7f053c661700 1 -- 172.21.15.32:0/2509761609 >> 172.21.15.32:0/2509761609 conn(0x7f05340fd8a0 msgr2=0x7f05340ffcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:38:02.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:02.834+0000 7f053c661700 1 -- 172.21.15.32:0/2509761609 shutdown_connections 2024-05-03T18:38:02.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:02.834+0000 7f053c661700 1 -- 172.21.15.32:0/2509761609 wait complete. 2024-05-03T18:38:02.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:02.835+0000 7f053c661700 1 Processor -- start 2024-05-03T18:38:02.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:02.835+0000 7f053c661700 1 -- start start 2024-05-03T18:38:02.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:02.835+0000 7f053c661700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0534104070 0x7f05341a07f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:38:02.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:02.835+0000 7f053c661700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f053410d410 con 0x7f0534104070 2024-05-03T18:38:02.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:02.835+0000 7f053a3fd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0534104070 0x7f05341a07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:38:02.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:02.835+0000 7f053a3fd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0534104070 0x7f05341a07f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45876/0 (socket says 172.21.15.32:45876) 2024-05-03T18:38:02.837 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:02.835+0000 7f053a3fd700 1 -- 172.21.15.32:0/1337228395 learned_addr learned my addr 172.21.15.32:0/1337228395 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:38:02.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:02.836+0000 7f053a3fd700 1 -- 172.21.15.32:0/1337228395 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05341a0d30 con 0x7f0534104070 2024-05-03T18:38:02.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:02.836+0000 7f053a3fd700 1 --2- 172.21.15.32:0/1337228395 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0534104070 0x7f05341a07f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f05300056d0 tx=0x7f05300057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:38:02.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:02.836+0000 7f052b7fe700 1 -- 172.21.15.32:0/1337228395 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0530026070 con 0x7f0534104070 2024-05-03T18:38:02.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:02.836+0000 7f052b7fe700 1 -- 172.21.15.32:0/1337228395 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0530004370 con 0x7f0534104070 2024-05-03T18:38:02.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:02.836+0000 7f053c661700 1 -- 172.21.15.32:0/1337228395 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05341a0fc0 con 0x7f0534104070 2024-05-03T18:38:02.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:02.836+0000 7f052b7fe700 1 -- 172.21.15.32:0/1337228395 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f05300039b0 con 0x7f0534104070 2024-05-03T18:38:02.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:02.836+0000 7f053c661700 1 -- 172.21.15.32:0/1337228395 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f053419a9b0 con 0x7f0534104070 2024-05-03T18:38:02.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:02.837+0000 7f052b7fe700 1 -- 172.21.15.32:0/1337228395 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f0530029070 con 0x7f0534104070 2024-05-03T18:38:02.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:02.837+0000 7f053c661700 1 -- 172.21.15.32:0/1337228395 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f053406bbf0 con 0x7f0534104070 2024-05-03T18:38:02.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:02.837+0000 7f052b7fe700 1 --2- 172.21.15.32:0/1337228395 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f052003e570 0x7f0520040a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:38:02.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:02.837+0000 7f052b7fe700 1 -- 172.21.15.32:0/1337228395 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f053001e070 con 0x7f0534104070 2024-05-03T18:38:02.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:02.837+0000 7f0539bfc700 1 --2- 172.21.15.32:0/1337228395 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f052003e570 0x7f0520040a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:38:02.839 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:02.838+0000 7f0539bfc700 1 --2- 172.21.15.32:0/1337228395 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f052003e570 0x7f0520040a30 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f0524006fd0 tx=0x7f0524006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:38:02.842 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:02.842+0000 7f052b7fe700 1 -- 172.21.15.32:0/1337228395 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f0530013070 con 0x7f0534104070 2024-05-03T18:38:03.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:03.060+0000 7f053c661700 1 -- 172.21.15.32:0/1337228395 --> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi045", "target": ["mon-mgr", ""]}) -- 0x7f053419bee0 con 0x7f052003e570 2024-05-03T18:38:03.817 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:03 smithi032 ceph-mon[29763]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi045", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:38:04.817 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:04 smithi032 ceph-mon[29763]: Deploying cephadm binary to smithi045 2024-05-03T18:38:07.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:06 smithi032 ceph-mon[29763]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:38:07.336 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:07.334+0000 7f052b7fe700 1 -- 172.21.15.32:0/1337228395 <== mgr.14152 v2:172.21.15.32:6800/1796547904 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f053419bee0 con 0x7f052003e570 2024-05-03T18:38:07.337 INFO:teuthology.orchestra.run.smithi032.stdout:Added host 'smithi045' with addr '172.21.15.45' 2024-05-03T18:38:07.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:07.338+0000 7f053c661700 1 -- 172.21.15.32:0/1337228395 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f052003e570 msgr2=0x7f0520040a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:38:07.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:07.338+0000 7f053c661700 1 --2- 172.21.15.32:0/1337228395 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f052003e570 0x7f0520040a30 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f0524006fd0 tx=0x7f0524006e40 comp rx=0 tx=0).stop 2024-05-03T18:38:07.339 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:07.338+0000 7f053c661700 1 -- 172.21.15.32:0/1337228395 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0534104070 msgr2=0x7f05341a07f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:38:07.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:07.338+0000 7f053c661700 1 --2- 172.21.15.32:0/1337228395 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0534104070 0x7f05341a07f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f05300056d0 tx=0x7f05300057b0 comp rx=0 tx=0).stop 2024-05-03T18:38:07.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:07.338+0000 7f053c661700 1 -- 172.21.15.32:0/1337228395 shutdown_connections 2024-05-03T18:38:07.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:07.338+0000 7f053c661700 1 --2- 172.21.15.32:0/1337228395 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f052003e570 0x7f0520040a30 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:07.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:07.338+0000 7f053c661700 1 --2- 172.21.15.32:0/1337228395 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0534104070 0x7f05341a07f0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:07.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:07.338+0000 7f053c661700 1 -- 172.21.15.32:0/1337228395 >> 172.21.15.32:0/1337228395 conn(0x7f05340fd8a0 msgr2=0x7f0534109230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:38:07.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:07.338+0000 7f053c661700 1 -- 172.21.15.32:0/1337228395 shutdown_connections 2024-05-03T18:38:07.340 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:07.338+0000 7f053c661700 1 -- 172.21.15.32:0/1337228395 wait complete. 2024-05-03T18:38:07.987 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f500455e-097b-11ef-bc97-c7b262605968 -- ceph orch host ls --format=json 2024-05-03T18:38:08.236 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:38:08.654 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:08 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:08.654 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:08 smithi032 ceph-mon[29763]: Added host smithi045 2024-05-03T18:38:08.654 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:08 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:38:08.654 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:08 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:09.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:09.697+0000 7f1792add700 1 -- 172.21.15.32:0/451856566 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f178c102010 msgr2=0x7f178c1023f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:38:09.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:09.697+0000 7f1792add700 1 --2- 172.21.15.32:0/451856566 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f178c102010 0x7f178c1023f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f1780009910 tx=0x7f1780009c20 comp rx=0 tx=0).stop 2024-05-03T18:38:09.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:09.697+0000 7f1792add700 1 -- 172.21.15.32:0/451856566 shutdown_connections 2024-05-03T18:38:09.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:09.697+0000 7f1792add700 1 --2- 172.21.15.32:0/451856566 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f178c102010 0x7f178c1023f0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:09.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:09.697+0000 7f1792add700 1 -- 172.21.15.32:0/451856566 >> 172.21.15.32:0/451856566 conn(0x7f178c0fd8a0 msgr2=0x7f178c0ffcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:38:09.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:09.697+0000 7f1792add700 1 -- 172.21.15.32:0/451856566 shutdown_connections 2024-05-03T18:38:09.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:09.697+0000 7f1792add700 1 -- 172.21.15.32:0/451856566 wait complete. 2024-05-03T18:38:09.700 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:09 smithi032 ceph-mon[29763]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:38:09.700 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:09 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:09.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:09.698+0000 7f1792add700 1 Processor -- start 2024-05-03T18:38:09.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:09.698+0000 7f1792add700 1 -- start start 2024-05-03T18:38:09.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:09.698+0000 7f1792add700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f178c102010 0x7f178c103ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:38:09.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:09.698+0000 7f1792add700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f178c10c290 con 0x7f178c102010 2024-05-03T18:38:09.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:09.698+0000 7f1790879700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f178c102010 0x7f178c103ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:38:09.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:09.698+0000 7f1790879700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f178c102010 0x7f178c103ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58184/0 (socket says 172.21.15.32:58184) 2024-05-03T18:38:09.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:09.698+0000 7f1790879700 1 -- 172.21.15.32:0/1775352130 learned_addr learned my addr 172.21.15.32:0/1775352130 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:38:09.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:09.698+0000 7f1790879700 1 -- 172.21.15.32:0/1775352130 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f178c1059a0 con 0x7f178c102010 2024-05-03T18:38:09.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:09.699+0000 7f1790879700 1 --2- 172.21.15.32:0/1775352130 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f178c102010 0x7f178c103ab0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f17800056d0 tx=0x7f17800057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:38:09.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:09.699+0000 7f1789ffb700 1 -- 172.21.15.32:0/1775352130 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1780026070 con 0x7f178c102010 2024-05-03T18:38:09.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:09.699+0000 7f1789ffb700 1 -- 172.21.15.32:0/1775352130 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f178000bc00 con 0x7f178c102010 2024-05-03T18:38:09.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:09.699+0000 7f1792add700 1 -- 172.21.15.32:0/1775352130 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f178c104050 con 0x7f178c102010 2024-05-03T18:38:09.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:09.699+0000 7f1789ffb700 1 -- 172.21.15.32:0/1775352130 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1780003920 con 0x7f178c102010 2024-05-03T18:38:09.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:09.699+0000 7f1792add700 1 -- 172.21.15.32:0/1775352130 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f178c104510 con 0x7f178c102010 2024-05-03T18:38:09.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:09.700+0000 7f1789ffb700 1 -- 172.21.15.32:0/1775352130 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f1780029070 con 0x7f178c102010 2024-05-03T18:38:09.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:09.700+0000 7f1792add700 1 -- 172.21.15.32:0/1775352130 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f178c1034f0 con 0x7f178c102010 2024-05-03T18:38:09.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:09.700+0000 7f1789ffb700 1 --2- 172.21.15.32:0/1775352130 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f177c03e570 0x7f177c040a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:38:09.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:09.700+0000 7f1789ffb700 1 -- 172.21.15.32:0/1775352130 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f178001e070 con 0x7f178c102010 2024-05-03T18:38:09.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:09.700+0000 7f178bfff700 1 --2- 172.21.15.32:0/1775352130 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f177c03e570 0x7f177c040a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:38:09.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:09.701+0000 7f178bfff700 1 --2- 172.21.15.32:0/1775352130 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f177c03e570 0x7f177c040a30 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f1778006fd0 tx=0x7f1778006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:38:09.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:09.705+0000 7f1789ffb700 1 -- 172.21.15.32:0/1775352130 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f1780013070 con 0x7f178c102010 2024-05-03T18:38:09.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:09.927+0000 7f1792add700 1 -- 172.21.15.32:0/1775352130 --> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f178c105050 con 0x7f177c03e570 2024-05-03T18:38:09.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:09.928+0000 7f1789ffb700 1 -- 172.21.15.32:0/1775352130 <== mgr.14152 v2:172.21.15.32:6800/1796547904 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+159 (secure 0 0 0) 0x7f178c105050 con 0x7f177c03e570 2024-05-03T18:38:09.929 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-03T18:38:09.930 INFO:teuthology.orchestra.run.smithi032.stdout:[{"addr": "172.21.15.32", "hostname": "smithi032", "labels": [], "status": ""}, {"addr": "172.21.15.45", "hostname": "smithi045", "labels": [], "status": ""}] 2024-05-03T18:38:09.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:09.930+0000 7f1792add700 1 -- 172.21.15.32:0/1775352130 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f177c03e570 msgr2=0x7f177c040a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:38:09.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:09.930+0000 7f1792add700 1 --2- 172.21.15.32:0/1775352130 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f177c03e570 0x7f177c040a30 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f1778006fd0 tx=0x7f1778006e40 comp rx=0 tx=0).stop 2024-05-03T18:38:09.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:09.930+0000 7f1792add700 1 -- 172.21.15.32:0/1775352130 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f178c102010 msgr2=0x7f178c103ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:38:09.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:09.930+0000 7f1792add700 1 --2- 172.21.15.32:0/1775352130 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f178c102010 0x7f178c103ab0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f17800056d0 tx=0x7f17800057b0 comp rx=0 tx=0).stop 2024-05-03T18:38:09.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:09.930+0000 7f1792add700 1 -- 172.21.15.32:0/1775352130 shutdown_connections 2024-05-03T18:38:09.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:09.930+0000 7f1792add700 1 --2- 172.21.15.32:0/1775352130 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f177c03e570 0x7f177c040a30 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:09.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:09.930+0000 7f1792add700 1 --2- 172.21.15.32:0/1775352130 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f178c102010 0x7f178c103ab0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:09.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:09.930+0000 7f1792add700 1 -- 172.21.15.32:0/1775352130 >> 172.21.15.32:0/1775352130 conn(0x7f178c0fd8a0 msgr2=0x7f178c06b310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:38:09.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:09.930+0000 7f1792add700 1 -- 172.21.15.32:0/1775352130 shutdown_connections 2024-05-03T18:38:09.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:09.930+0000 7f1792add700 1 -- 172.21.15.32:0/1775352130 wait complete. 2024-05-03T18:38:10.577 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi113 2024-05-03T18:38:10.577 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-05-03T18:38:10.577 DEBUG:teuthology.orchestra.run.smithi113:> dd of=/etc/ceph/ceph.conf 2024-05-03T18:38:10.596 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-05-03T18:38:10.596 DEBUG:teuthology.orchestra.run.smithi113:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-03T18:38:10.653 INFO:tasks.cephadm:Adding host smithi113 to orchestrator... 2024-05-03T18:38:10.653 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f500455e-097b-11ef-bc97-c7b262605968 -- ceph orch host add smithi113 2024-05-03T18:38:10.897 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:38:11.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:11 smithi032 ceph-mon[29763]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:38:11.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:11 smithi032 ceph-mon[29763]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-03T18:38:11.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:11 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:12.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:12.309+0000 7f6f84f04700 1 -- 172.21.15.32:0/3125675041 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f801075a0 msgr2=0x7f6f80107980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:38:12.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:12.309+0000 7f6f84f04700 1 --2- 172.21.15.32:0/3125675041 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f801075a0 0x7f6f80107980 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f6f7000aa20 tx=0x7f6f7000ad30 comp rx=0 tx=0).stop 2024-05-03T18:38:12.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:12.309+0000 7f6f84f04700 1 -- 172.21.15.32:0/3125675041 shutdown_connections 2024-05-03T18:38:12.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:12.309+0000 7f6f84f04700 1 --2- 172.21.15.32:0/3125675041 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f801075a0 0x7f6f80107980 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:12.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:12.309+0000 7f6f84f04700 1 -- 172.21.15.32:0/3125675041 >> 172.21.15.32:0/3125675041 conn(0x7f6f800793d0 msgr2=0x7f6f800797e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:38:12.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:12.309+0000 7f6f84f04700 1 -- 172.21.15.32:0/3125675041 shutdown_connections 2024-05-03T18:38:12.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:12.310+0000 7f6f84f04700 1 -- 172.21.15.32:0/3125675041 wait complete. 2024-05-03T18:38:12.313 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:12.310+0000 7f6f84f04700 1 Processor -- start 2024-05-03T18:38:12.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:12.310+0000 7f6f84f04700 1 -- start start 2024-05-03T18:38:12.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:12.311+0000 7f6f84f04700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f801075a0 0x7f6f8019e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:38:12.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:12.311+0000 7f6f84f04700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f6f8010b180 con 0x7f6f801075a0 2024-05-03T18:38:12.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:12.311+0000 7f6f7e59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f801075a0 0x7f6f8019e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:38:12.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:12.311+0000 7f6f7e59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f801075a0 0x7f6f8019e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58202/0 (socket says 172.21.15.32:58202) 2024-05-03T18:38:12.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:12.311+0000 7f6f7e59c700 1 -- 172.21.15.32:0/3543842378 learned_addr learned my addr 172.21.15.32:0/3543842378 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:38:12.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:12.311+0000 7f6f7e59c700 1 -- 172.21.15.32:0/3543842378 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f8019e930 con 0x7f6f801075a0 2024-05-03T18:38:12.314 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:12.311+0000 7f6f7e59c700 1 --2- 172.21.15.32:0/3543842378 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f801075a0 0x7f6f8019e3f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f6f70006010 tx=0x7f6f70005050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:38:12.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:12.311+0000 7f6f6f7fe700 1 -- 172.21.15.32:0/3543842378 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6f70005400 con 0x7f6f801075a0 2024-05-03T18:38:12.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:12.311+0000 7f6f6f7fe700 1 -- 172.21.15.32:0/3543842378 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f700055a0 con 0x7f6f801075a0 2024-05-03T18:38:12.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:12.311+0000 7f6f84f04700 1 -- 172.21.15.32:0/3543842378 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f8019ebc0 con 0x7f6f801075a0 2024-05-03T18:38:12.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:12.311+0000 7f6f6f7fe700 1 -- 172.21.15.32:0/3543842378 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6f7000e700 con 0x7f6f801075a0 2024-05-03T18:38:12.315 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:12.311+0000 7f6f84f04700 1 -- 172.21.15.32:0/3543842378 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f801985b0 con 0x7f6f801075a0 2024-05-03T18:38:12.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:12.312+0000 7f6f6f7fe700 1 -- 172.21.15.32:0/3543842378 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f6f7000e8a0 con 0x7f6f801075a0 2024-05-03T18:38:12.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:12.312+0000 7f6f84f04700 1 -- 172.21.15.32:0/3543842378 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f80102c80 con 0x7f6f801075a0 2024-05-03T18:38:12.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:12.312+0000 7f6f6f7fe700 1 --2- 172.21.15.32:0/3543842378 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f6f6803e570 0x7f6f68040a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:38:12.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:12.312+0000 7f6f6f7fe700 1 -- 172.21.15.32:0/3543842378 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f6f7001e070 con 0x7f6f801075a0 2024-05-03T18:38:12.317 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:12.313+0000 7f6f7dd9b700 1 --2- 172.21.15.32:0/3543842378 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f6f6803e570 0x7f6f68040a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:38:12.318 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:12.313+0000 7f6f7dd9b700 1 --2- 172.21.15.32:0/3543842378 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f6f6803e570 0x7f6f68040a30 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f6f74006fd0 tx=0x7f6f74006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:38:12.319 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:12.318+0000 7f6f6f7fe700 1 -- 172.21.15.32:0/3543842378 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f6f70020df0 con 0x7f6f801075a0 2024-05-03T18:38:12.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:12.537+0000 7f6f84f04700 1 -- 172.21.15.32:0/3543842378 --> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi113", "target": ["mon-mgr", ""]}) -- 0x7f6f80199ae0 con 0x7f6f6803e570 2024-05-03T18:38:13.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:13 smithi032 ceph-mon[29763]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:38:14.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:14 smithi032 ceph-mon[29763]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi113", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:38:14.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:14 smithi032 ceph-mon[29763]: Deploying cephadm binary to smithi113 2024-05-03T18:38:14.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:14 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:14.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:14 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:14.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:14 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:14.569 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:14 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:14.569 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:14 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi045", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:38:14.569 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:14 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:38:14.569 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:14 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:38:14.569 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:14 smithi032 ceph-mon[29763]: Updating smithi045:/etc/ceph/ceph.conf 2024-05-03T18:38:15.567 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:15 smithi032 ceph-mon[29763]: Updating smithi045:/var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-03T18:38:15.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:15 smithi032 ceph-mon[29763]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:38:16.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:16.734+0000 7f6f6f7fe700 1 -- 172.21.15.32:0/3543842378 <== mgr.14152 v2:172.21.15.32:6800/1796547904 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f6f80199ae0 con 0x7f6f6803e570 2024-05-03T18:38:16.736 INFO:teuthology.orchestra.run.smithi032.stdout:Added host 'smithi113' with addr '172.21.15.113' 2024-05-03T18:38:16.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:16.737+0000 7f6f84f04700 1 -- 172.21.15.32:0/3543842378 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f6f6803e570 msgr2=0x7f6f68040a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:38:16.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:16.737+0000 7f6f84f04700 1 --2- 172.21.15.32:0/3543842378 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f6f6803e570 0x7f6f68040a30 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f6f74006fd0 tx=0x7f6f74006e40 comp rx=0 tx=0).stop 2024-05-03T18:38:16.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:16.737+0000 7f6f84f04700 1 -- 172.21.15.32:0/3543842378 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f801075a0 msgr2=0x7f6f8019e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:38:16.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:16.737+0000 7f6f84f04700 1 --2- 172.21.15.32:0/3543842378 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f801075a0 0x7f6f8019e3f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f6f70006010 tx=0x7f6f70005050 comp rx=0 tx=0).stop 2024-05-03T18:38:16.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:16.737+0000 7f6f84f04700 1 -- 172.21.15.32:0/3543842378 shutdown_connections 2024-05-03T18:38:16.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:16.737+0000 7f6f84f04700 1 --2- 172.21.15.32:0/3543842378 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f6f6803e570 0x7f6f68040a30 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:16.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:16.737+0000 7f6f84f04700 1 --2- 172.21.15.32:0/3543842378 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f801075a0 0x7f6f8019e3f0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:16.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:16.737+0000 7f6f84f04700 1 -- 172.21.15.32:0/3543842378 >> 172.21.15.32:0/3543842378 conn(0x7f6f800793d0 msgr2=0x7f6f80100240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:38:16.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:16.737+0000 7f6f84f04700 1 -- 172.21.15.32:0/3543842378 shutdown_connections 2024-05-03T18:38:16.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:16.737+0000 7f6f84f04700 1 -- 172.21.15.32:0/3543842378 wait complete. 2024-05-03T18:38:17.301 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:17 smithi032 ceph-mon[29763]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:38:17.301 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:17 smithi032 ceph-mon[29763]: Updating smithi045:/etc/ceph/ceph.client.admin.keyring 2024-05-03T18:38:17.301 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:17 smithi032 ceph-mon[29763]: Updating smithi045:/var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-03T18:38:17.301 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:17 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:17.301 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:17 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:17.301 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:17 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:17.301 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:17 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:17.301 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:17 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:38:17.301 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:17 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:17.329 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f500455e-097b-11ef-bc97-c7b262605968 -- ceph orch host ls --format=json 2024-05-03T18:38:17.569 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:38:18.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:18 smithi032 ceph-mon[29763]: Added host smithi113 2024-05-03T18:38:18.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:18 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:19.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:19.148+0000 7f36648a0700 1 -- 172.21.15.32:0/4128022137 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3660108f40 msgr2=0x7f3660109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:38:19.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:19.148+0000 7f36648a0700 1 --2- 172.21.15.32:0/4128022137 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3660108f40 0x7f3660109320 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f3650009910 tx=0x7f3650009c20 comp rx=0 tx=0).stop 2024-05-03T18:38:19.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:19.148+0000 7f36648a0700 1 -- 172.21.15.32:0/4128022137 shutdown_connections 2024-05-03T18:38:19.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:19.148+0000 7f36648a0700 1 --2- 172.21.15.32:0/4128022137 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3660108f40 0x7f3660109320 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:19.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:19.148+0000 7f36648a0700 1 -- 172.21.15.32:0/4128022137 >> 172.21.15.32:0/4128022137 conn(0x7f3660077900 msgr2=0x7f3660077d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:38:19.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:19.148+0000 7f36648a0700 1 -- 172.21.15.32:0/4128022137 shutdown_connections 2024-05-03T18:38:19.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:19.148+0000 7f36648a0700 1 -- 172.21.15.32:0/4128022137 wait complete. 2024-05-03T18:38:19.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:19.149+0000 7f36648a0700 1 Processor -- start 2024-05-03T18:38:19.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:19.149+0000 7f36648a0700 1 -- start start 2024-05-03T18:38:19.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:19.149+0000 7f36648a0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3660108f40 0x7f366019e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:38:19.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:19.149+0000 7f36648a0700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f366010d380 con 0x7f3660108f40 2024-05-03T18:38:19.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:19.149+0000 7f365e59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3660108f40 0x7f366019e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:38:19.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:19.149+0000 7f365e59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3660108f40 0x7f366019e650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58908/0 (socket says 172.21.15.32:58908) 2024-05-03T18:38:19.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:19.149+0000 7f365e59c700 1 -- 172.21.15.32:0/1265509165 learned_addr learned my addr 172.21.15.32:0/1265509165 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:38:19.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:19.149+0000 7f365e59c700 1 -- 172.21.15.32:0/1265509165 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f366019eb90 con 0x7f3660108f40 2024-05-03T18:38:19.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:19.150+0000 7f365e59c700 1 --2- 172.21.15.32:0/1265509165 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3660108f40 0x7f366019e650 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f3650006010 tx=0x7f3650005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:38:19.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:19.150+0000 7f364f7fe700 1 -- 172.21.15.32:0/1265509165 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3650026070 con 0x7f3660108f40 2024-05-03T18:38:19.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:19.150+0000 7f364f7fe700 1 -- 172.21.15.32:0/1265509165 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f365000bba0 con 0x7f3660108f40 2024-05-03T18:38:19.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:19.150+0000 7f36648a0700 1 -- 172.21.15.32:0/1265509165 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f366019ee80 con 0x7f3660108f40 2024-05-03T18:38:19.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:19.150+0000 7f364f7fe700 1 -- 172.21.15.32:0/1265509165 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3650003910 con 0x7f3660108f40 2024-05-03T18:38:19.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:19.150+0000 7f36648a0700 1 -- 172.21.15.32:0/1265509165 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36601988f0 con 0x7f3660108f40 2024-05-03T18:38:19.153 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:19.151+0000 7f364f7fe700 1 -- 172.21.15.32:0/1265509165 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f3650029070 con 0x7f3660108f40 2024-05-03T18:38:19.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:19.151+0000 7f36648a0700 1 -- 172.21.15.32:0/1265509165 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36601045f0 con 0x7f3660108f40 2024-05-03T18:38:19.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:19.151+0000 7f364f7fe700 1 --2- 172.21.15.32:0/1265509165 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f364803e510 0x7f36480409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:38:19.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:19.151+0000 7f364f7fe700 1 -- 172.21.15.32:0/1265509165 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f365001e070 con 0x7f3660108f40 2024-05-03T18:38:19.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:19.151+0000 7f365dd9b700 1 --2- 172.21.15.32:0/1265509165 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f364803e510 0x7f36480409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:38:19.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:19.152+0000 7f365dd9b700 1 --2- 172.21.15.32:0/1265509165 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f364803e510 0x7f36480409d0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f3654006fd0 tx=0x7f3654006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:38:19.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:19.156+0000 7f364f7fe700 1 -- 172.21.15.32:0/1265509165 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f3650013070 con 0x7f3660108f40 2024-05-03T18:38:19.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:19.376+0000 7f36648a0700 1 -- 172.21.15.32:0/1265509165 --> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3660199cc0 con 0x7f364803e510 2024-05-03T18:38:19.377 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:19.377+0000 7f364f7fe700 1 -- 172.21.15.32:0/1265509165 <== mgr.14152 v2:172.21.15.32:6800/1796547904 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+239 (secure 0 0 0) 0x7f3660199cc0 con 0x7f364803e510 2024-05-03T18:38:19.377 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-03T18:38:19.378 INFO:teuthology.orchestra.run.smithi032.stdout:[{"addr": "172.21.15.32", "hostname": "smithi032", "labels": [], "status": ""}, {"addr": "172.21.15.45", "hostname": "smithi045", "labels": [], "status": ""}, {"addr": "172.21.15.113", "hostname": "smithi113", "labels": [], "status": ""}] 2024-05-03T18:38:19.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:19.379+0000 7f36648a0700 1 -- 172.21.15.32:0/1265509165 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f364803e510 msgr2=0x7f36480409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:38:19.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:19.379+0000 7f36648a0700 1 --2- 172.21.15.32:0/1265509165 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f364803e510 0x7f36480409d0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f3654006fd0 tx=0x7f3654006e40 comp rx=0 tx=0).stop 2024-05-03T18:38:19.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:19.379+0000 7f36648a0700 1 -- 172.21.15.32:0/1265509165 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3660108f40 msgr2=0x7f366019e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:38:19.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:19.379+0000 7f36648a0700 1 --2- 172.21.15.32:0/1265509165 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3660108f40 0x7f366019e650 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f3650006010 tx=0x7f3650005780 comp rx=0 tx=0).stop 2024-05-03T18:38:19.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:19.379+0000 7f36648a0700 1 -- 172.21.15.32:0/1265509165 shutdown_connections 2024-05-03T18:38:19.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:19.379+0000 7f36648a0700 1 --2- 172.21.15.32:0/1265509165 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f364803e510 0x7f36480409d0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:19.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:19.379+0000 7f36648a0700 1 --2- 172.21.15.32:0/1265509165 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3660108f40 0x7f366019e650 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:19.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:19.379+0000 7f36648a0700 1 -- 172.21.15.32:0/1265509165 >> 172.21.15.32:0/1265509165 conn(0x7f3660077900 msgr2=0x7f36600ff340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:38:19.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:19.379+0000 7f36648a0700 1 -- 172.21.15.32:0/1265509165 shutdown_connections 2024-05-03T18:38:19.380 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:19.379+0000 7f36648a0700 1 -- 172.21.15.32:0/1265509165 wait complete. 2024-05-03T18:38:19.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:19 smithi032 ceph-mon[29763]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:38:19.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:19 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:20.027 INFO:tasks.cephadm:Setting crush tunables to default 2024-05-03T18:38:20.027 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f500455e-097b-11ef-bc97-c7b262605968 -- ceph osd crush tunables default 2024-05-03T18:38:20.271 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:38:20.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:20 smithi032 ceph-mon[29763]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-03T18:38:21.510 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:21 smithi032 ceph-mon[29763]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:38:21.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:21.729+0000 7f3f7ac9b700 1 -- 172.21.15.32:0/2701496596 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f74102010 msgr2=0x7f3f741023f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:38:21.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:21.729+0000 7f3f7ac9b700 1 --2- 172.21.15.32:0/2701496596 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f74102010 0x7f3f741023f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3f64009910 tx=0x7f3f64009c20 comp rx=0 tx=0).stop 2024-05-03T18:38:21.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:21.729+0000 7f3f7ac9b700 1 -- 172.21.15.32:0/2701496596 shutdown_connections 2024-05-03T18:38:21.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:21.729+0000 7f3f7ac9b700 1 --2- 172.21.15.32:0/2701496596 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f74102010 0x7f3f741023f0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:21.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:21.729+0000 7f3f7ac9b700 1 -- 172.21.15.32:0/2701496596 >> 172.21.15.32:0/2701496596 conn(0x7f3f740fd8a0 msgr2=0x7f3f740ffcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:38:21.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:21.729+0000 7f3f7ac9b700 1 -- 172.21.15.32:0/2701496596 shutdown_connections 2024-05-03T18:38:21.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:21.729+0000 7f3f7ac9b700 1 -- 172.21.15.32:0/2701496596 wait complete. 2024-05-03T18:38:21.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:21.729+0000 7f3f7ac9b700 1 Processor -- start 2024-05-03T18:38:21.735 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:21.730+0000 7f3f7ac9b700 1 -- start start 2024-05-03T18:38:21.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:21.730+0000 7f3f7ac9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f74102010 0x7f3f7419e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:38:21.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:21.730+0000 7f3f7ac9b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f3f7410c290 con 0x7f3f74102010 2024-05-03T18:38:21.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:21.730+0000 7f3f78a37700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f74102010 0x7f3f7419e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:38:21.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:21.730+0000 7f3f78a37700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f74102010 0x7f3f7419e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58926/0 (socket says 172.21.15.32:58926) 2024-05-03T18:38:21.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:21.730+0000 7f3f78a37700 1 -- 172.21.15.32:0/566430011 learned_addr learned my addr 172.21.15.32:0/566430011 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:38:21.736 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:21.730+0000 7f3f78a37700 1 -- 172.21.15.32:0/566430011 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f7419eb60 con 0x7f3f74102010 2024-05-03T18:38:21.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:21.730+0000 7f3f78a37700 1 --2- 172.21.15.32:0/566430011 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f74102010 0x7f3f7419e620 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f3f640056d0 tx=0x7f3f640057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:38:21.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:21.730+0000 7f3f71ffb700 1 -- 172.21.15.32:0/566430011 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3f64026070 con 0x7f3f74102010 2024-05-03T18:38:21.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:21.730+0000 7f3f71ffb700 1 -- 172.21.15.32:0/566430011 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f6400bba0 con 0x7f3f74102010 2024-05-03T18:38:21.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:21.730+0000 7f3f7ac9b700 1 -- 172.21.15.32:0/566430011 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f7419ee50 con 0x7f3f74102010 2024-05-03T18:38:21.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:21.731+0000 7f3f71ffb700 1 -- 172.21.15.32:0/566430011 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3f64003920 con 0x7f3f74102010 2024-05-03T18:38:21.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:21.731+0000 7f3f7ac9b700 1 -- 172.21.15.32:0/566430011 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f741988c0 con 0x7f3f74102010 2024-05-03T18:38:21.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:21.732+0000 7f3f71ffb700 1 -- 172.21.15.32:0/566430011 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f3f64029070 con 0x7f3f74102010 2024-05-03T18:38:21.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:21.732+0000 7f3f7ac9b700 1 -- 172.21.15.32:0/566430011 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f741034f0 con 0x7f3f74102010 2024-05-03T18:38:21.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:21.732+0000 7f3f71ffb700 1 --2- 172.21.15.32:0/566430011 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f3f5c03e510 0x7f3f5c0409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:38:21.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:21.732+0000 7f3f71ffb700 1 -- 172.21.15.32:0/566430011 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f3f6401e070 con 0x7f3f74102010 2024-05-03T18:38:21.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:21.732+0000 7f3f73fff700 1 --2- 172.21.15.32:0/566430011 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f3f5c03e510 0x7f3f5c0409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:38:21.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:21.732+0000 7f3f73fff700 1 --2- 172.21.15.32:0/566430011 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f3f5c03e510 0x7f3f5c0409d0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f3f68006fd0 tx=0x7f3f68006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:38:21.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:21.737+0000 7f3f71ffb700 1 -- 172.21.15.32:0/566430011 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f3f64013070 con 0x7f3f74102010 2024-05-03T18:38:21.960 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:21.959+0000 7f3f7ac9b700 1 -- 172.21.15.32:0/566430011 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f3f740008d0 con 0x7f3f74102010 2024-05-03T18:38:22.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:22.805+0000 7f3f71ffb700 1 -- 172.21.15.32:0/566430011 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7f3f6400bd50 con 0x7f3f74102010 2024-05-03T18:38:22.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:22.808+0000 7f3f7ac9b700 1 -- 172.21.15.32:0/566430011 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f3f5c03e510 msgr2=0x7f3f5c0409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:38:22.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:22.808+0000 7f3f7ac9b700 1 --2- 172.21.15.32:0/566430011 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f3f5c03e510 0x7f3f5c0409d0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f3f68006fd0 tx=0x7f3f68006e40 comp rx=0 tx=0).stop 2024-05-03T18:38:22.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:22.808+0000 7f3f7ac9b700 1 -- 172.21.15.32:0/566430011 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f74102010 msgr2=0x7f3f7419e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:38:22.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:22.808+0000 7f3f7ac9b700 1 --2- 172.21.15.32:0/566430011 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f74102010 0x7f3f7419e620 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f3f640056d0 tx=0x7f3f640057b0 comp rx=0 tx=0).stop 2024-05-03T18:38:22.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:22.808+0000 7f3f7ac9b700 1 -- 172.21.15.32:0/566430011 shutdown_connections 2024-05-03T18:38:22.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:22.808+0000 7f3f7ac9b700 1 --2- 172.21.15.32:0/566430011 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f3f5c03e510 0x7f3f5c0409d0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:22.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:22.808+0000 7f3f7ac9b700 1 --2- 172.21.15.32:0/566430011 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f74102010 0x7f3f7419e620 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:22.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:22.808+0000 7f3f7ac9b700 1 -- 172.21.15.32:0/566430011 >> 172.21.15.32:0/566430011 conn(0x7f3f740fd8a0 msgr2=0x7f3f7406b2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:38:22.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:22.808+0000 7f3f7ac9b700 1 -- 172.21.15.32:0/566430011 shutdown_connections 2024-05-03T18:38:22.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:22.808+0000 7f3f7ac9b700 1 -- 172.21.15.32:0/566430011 wait complete. 2024-05-03T18:38:22.810 INFO:teuthology.orchestra.run.smithi032.stderr:adjusted tunables profile to default 2024-05-03T18:38:23.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:22 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:23.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:22 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:23.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:22 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:23.069 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:22 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:23.069 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:22 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi113", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:38:23.069 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:22 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:38:23.069 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:22 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:38:23.069 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:22 smithi032 ceph-mon[29763]: Updating smithi113:/etc/ceph/ceph.conf 2024-05-03T18:38:23.069 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:22 smithi032 ceph-mon[29763]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:38:23.069 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:22 smithi032 ceph-mon[29763]: from='client.? 172.21.15.32:0/566430011' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-05-03T18:38:23.069 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:22 smithi032 ceph-mon[29763]: Updating smithi113:/var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-03T18:38:23.069 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:22 smithi032 ceph-mon[29763]: Updating smithi113:/etc/ceph/ceph.client.admin.keyring 2024-05-03T18:38:23.070 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:22 smithi032 ceph-mon[29763]: Updating smithi113:/var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-03T18:38:23.070 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:22 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:23.070 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:22 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:23.070 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:22 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:23.424 INFO:tasks.cephadm:Adding mon.a on smithi032 2024-05-03T18:38:23.424 INFO:tasks.cephadm:Adding mon.b on smithi045 2024-05-03T18:38:23.424 INFO:tasks.cephadm:Adding mon.c on smithi113 2024-05-03T18:38:23.424 DEBUG:teuthology.orchestra.run.smithi113:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f500455e-097b-11ef-bc97-c7b262605968 -- ceph orch apply mon '3;smithi032:172.21.15.32=a;smithi045:172.21.15.45=b;smithi113:172.21.15.113=c' 2024-05-03T18:38:23.660 INFO:teuthology.orchestra.run.smithi113.stderr:Inferring config /var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-03T18:38:24.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:23 smithi032 ceph-mon[29763]: from='client.? 172.21.15.32:0/566430011' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-05-03T18:38:24.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:23 smithi032 ceph-mon[29763]: osdmap e4: 0 total, 0 up, 0 in 2024-05-03T18:38:25.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:24 smithi032 ceph-mon[29763]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:38:25.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:25.123+0000 7fddcc29c700 1 -- 172.21.15.113:0/376908961 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fddc4101ff0 msgr2=0x7fddc41023d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:38:25.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:25.123+0000 7fddcc29c700 1 --2- 172.21.15.113:0/376908961 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fddc4101ff0 0x7fddc41023d0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fddc000b0a0 tx=0x7fddc000b3b0 comp rx=0 tx=0).stop 2024-05-03T18:38:25.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:25.123+0000 7fddcc29c700 1 -- 172.21.15.113:0/376908961 shutdown_connections 2024-05-03T18:38:25.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:25.123+0000 7fddcc29c700 1 --2- 172.21.15.113:0/376908961 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fddc4101ff0 0x7fddc41023d0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:25.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:25.123+0000 7fddcc29c700 1 -- 172.21.15.113:0/376908961 >> 172.21.15.113:0/376908961 conn(0x7fddc40fd8a0 msgr2=0x7fddc40ffcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:38:25.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:25.124+0000 7fddcc29c700 1 -- 172.21.15.113:0/376908961 shutdown_connections 2024-05-03T18:38:25.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:25.124+0000 7fddcc29c700 1 -- 172.21.15.113:0/376908961 wait complete. 2024-05-03T18:38:25.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:25.124+0000 7fddcc29c700 1 Processor -- start 2024-05-03T18:38:25.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:25.124+0000 7fddcc29c700 1 -- start start 2024-05-03T18:38:25.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:25.124+0000 7fddcc29c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fddc4101ff0 0x7fddc419e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:38:25.125 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:25.124+0000 7fddcc29c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fddc410c270 con 0x7fddc4101ff0 2024-05-03T18:38:25.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:25.125+0000 7fddca038700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fddc4101ff0 0x7fddc419e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:38:25.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:25.125+0000 7fddca038700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fddc4101ff0 0x7fddc419e5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.113:50686/0 (socket says 172.21.15.113:50686) 2024-05-03T18:38:25.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:25.125+0000 7fddca038700 1 -- 172.21.15.113:0/2440773239 learned_addr learned my addr 172.21.15.113:0/2440773239 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-05-03T18:38:25.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:25.125+0000 7fddca038700 1 -- 172.21.15.113:0/2440773239 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddc419eb00 con 0x7fddc4101ff0 2024-05-03T18:38:25.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:25.125+0000 7fddca038700 1 --2- 172.21.15.113:0/2440773239 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fddc4101ff0 0x7fddc419e5c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fddc000d010 tx=0x7fddc0003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:38:25.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:25.125+0000 7fddbb7fe700 1 -- 172.21.15.113:0/2440773239 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fddc0021020 con 0x7fddc4101ff0 2024-05-03T18:38:25.127 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:25.125+0000 7fddbb7fe700 1 -- 172.21.15.113:0/2440773239 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fddc00044b0 con 0x7fddc4101ff0 2024-05-03T18:38:25.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:25.125+0000 7fddcc29c700 1 -- 172.21.15.113:0/2440773239 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddc419ed90 con 0x7fddc4101ff0 2024-05-03T18:38:25.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:25.125+0000 7fddbb7fe700 1 -- 172.21.15.113:0/2440773239 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fddc0017040 con 0x7fddc4101ff0 2024-05-03T18:38:25.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:25.125+0000 7fddcc29c700 1 -- 172.21.15.113:0/2440773239 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddc4198690 con 0x7fddc4101ff0 2024-05-03T18:38:25.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:25.126+0000 7fddcc29c700 1 -- 172.21.15.113:0/2440773239 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fddc41034d0 con 0x7fddc4101ff0 2024-05-03T18:38:25.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:25.126+0000 7fddbb7fe700 1 -- 172.21.15.113:0/2440773239 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7fddc0024020 con 0x7fddc4101ff0 2024-05-03T18:38:25.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:25.126+0000 7fddbb7fe700 1 --2- 172.21.15.113:0/2440773239 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fddb003e560 0x7fddb0040a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:38:25.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:25.127+0000 7fddbb7fe700 1 -- 172.21.15.113:0/2440773239 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fddc000f050 con 0x7fddc4101ff0 2024-05-03T18:38:25.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:25.127+0000 7fddc9837700 1 --2- 172.21.15.113:0/2440773239 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fddb003e560 0x7fddb0040a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:38:25.128 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:25.127+0000 7fddc9837700 1 --2- 172.21.15.113:0/2440773239 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fddb003e560 0x7fddb0040a20 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fddb4006fd0 tx=0x7fddb4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:38:25.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:25.132+0000 7fddbb7fe700 1 -- 172.21.15.113:0/2440773239 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fddc002b020 con 0x7fddc4101ff0 2024-05-03T18:38:25.347 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:25.346+0000 7fddcc29c700 1 -- 172.21.15.113:0/2440773239 --> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi032:172.21.15.32=a;smithi045:172.21.15.45=b;smithi113:172.21.15.113=c", "target": ["mon-mgr", ""]}) -- 0x7fddc4199bc0 con 0x7fddb003e560 2024-05-03T18:38:25.353 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:25.352+0000 7fddbb7fe700 1 -- 172.21.15.113:0/2440773239 <== mgr.14152 v2:172.21.15.32:6800/1796547904 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fddc4199bc0 con 0x7fddb003e560 2024-05-03T18:38:25.353 INFO:teuthology.orchestra.run.smithi113.stdout:Scheduled mon update... 2024-05-03T18:38:25.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:25.355+0000 7fddcc29c700 1 -- 172.21.15.113:0/2440773239 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fddb003e560 msgr2=0x7fddb0040a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:38:25.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:25.355+0000 7fddcc29c700 1 --2- 172.21.15.113:0/2440773239 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fddb003e560 0x7fddb0040a20 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fddb4006fd0 tx=0x7fddb4006e40 comp rx=0 tx=0).stop 2024-05-03T18:38:25.356 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:25.355+0000 7fddcc29c700 1 -- 172.21.15.113:0/2440773239 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fddc4101ff0 msgr2=0x7fddc419e5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:38:25.356 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:25.355+0000 7fddcc29c700 1 --2- 172.21.15.113:0/2440773239 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fddc4101ff0 0x7fddc419e5c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fddc000d010 tx=0x7fddc0003ce0 comp rx=0 tx=0).stop 2024-05-03T18:38:25.356 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:25.355+0000 7fddcc29c700 1 -- 172.21.15.113:0/2440773239 shutdown_connections 2024-05-03T18:38:25.356 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:25.355+0000 7fddcc29c700 1 --2- 172.21.15.113:0/2440773239 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fddb003e560 0x7fddb0040a20 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:25.356 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:25.355+0000 7fddcc29c700 1 --2- 172.21.15.113:0/2440773239 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fddc4101ff0 0x7fddc419e5c0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:25.356 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:25.355+0000 7fddcc29c700 1 -- 172.21.15.113:0/2440773239 >> 172.21.15.113:0/2440773239 conn(0x7fddc40fd8a0 msgr2=0x7fddc406b130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:38:25.356 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:25.355+0000 7fddcc29c700 1 -- 172.21.15.113:0/2440773239 shutdown_connections 2024-05-03T18:38:25.356 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:25.355+0000 7fddcc29c700 1 -- 172.21.15.113:0/2440773239 wait complete. 2024-05-03T18:38:25.961 DEBUG:teuthology.orchestra.run.smithi045:mon.b> sudo journalctl -f -n 0 -u ceph-f500455e-097b-11ef-bc97-c7b262605968@mon.b.service 2024-05-03T18:38:25.963 DEBUG:teuthology.orchestra.run.smithi113:mon.c> sudo journalctl -f -n 0 -u ceph-f500455e-097b-11ef-bc97-c7b262605968@mon.c.service 2024-05-03T18:38:25.966 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-03T18:38:25.966 DEBUG:teuthology.orchestra.run.smithi113:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f500455e-097b-11ef-bc97-c7b262605968 -- ceph mon dump -f json 2024-05-03T18:38:26.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:26 smithi032 ceph-mon[29763]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi032:172.21.15.32=a;smithi045:172.21.15.45=b;smithi113:172.21.15.113=c", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:38:26.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:26 smithi032 ceph-mon[29763]: Saving service mon spec with placement smithi032:172.21.15.32=a;smithi045:172.21.15.45=b;smithi113:172.21.15.113=c;count:3 2024-05-03T18:38:26.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:26 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:26.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:26 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:38:26.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:26 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:38:26.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:26 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:38:26.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:26 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:26.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:26 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-03T18:38:26.819 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:26 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:38:26.819 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:26 smithi032 ceph-mon[29763]: Deploying daemon mon.c on smithi113 2024-05-03T18:38:27.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:27 smithi032 ceph-mon[29763]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:38:28.169 INFO:teuthology.orchestra.run.smithi113.stderr:Inferring config /var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/mon.c/config 2024-05-03T18:38:29.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:29 smithi032 ceph-mon[29763]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:38:30.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:30.027+0000 7f0f7e65e700 1 -- 172.21.15.113:0/2229621559 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0f78103240 msgr2=0x7f0f78103620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:38:30.236 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:30.027+0000 7f0f7e65e700 1 --2- 172.21.15.113:0/2229621559 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0f78103240 0x7f0f78103620 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f0f68009910 tx=0x7f0f68009c20 comp rx=0 tx=0).stop 2024-05-03T18:38:30.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:30.027+0000 7f0f7e65e700 1 -- 172.21.15.113:0/2229621559 shutdown_connections 2024-05-03T18:38:30.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:30.027+0000 7f0f7e65e700 1 --2- 172.21.15.113:0/2229621559 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0f78103240 0x7f0f78103620 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:30.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:30.027+0000 7f0f7e65e700 1 -- 172.21.15.113:0/2229621559 >> 172.21.15.113:0/2229621559 conn(0x7f0f780fea90 msgr2=0x7f0f78100eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:38:30.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:30.027+0000 7f0f7e65e700 1 -- 172.21.15.113:0/2229621559 shutdown_connections 2024-05-03T18:38:30.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:30.027+0000 7f0f7e65e700 1 -- 172.21.15.113:0/2229621559 wait complete. 2024-05-03T18:38:30.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:30.028+0000 7f0f7e65e700 1 Processor -- start 2024-05-03T18:38:30.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:30.028+0000 7f0f7e65e700 1 -- start start 2024-05-03T18:38:30.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:30.028+0000 7f0f7e65e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0f78103240 0x7f0f7819e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:38:30.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:30.028+0000 7f0f7e65e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f0f7810d4c0 con 0x7f0f78103240 2024-05-03T18:38:30.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:30.029+0000 7f0f77fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0f78103240 0x7f0f7819e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:38:30.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:30.029+0000 7f0f77fff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0f78103240 0x7f0f7819e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.113:33368/0 (socket says 172.21.15.113:33368) 2024-05-03T18:38:30.237 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:30.029+0000 7f0f77fff700 1 -- 172.21.15.113:0/417409240 learned_addr learned my addr 172.21.15.113:0/417409240 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-05-03T18:38:30.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:30.029+0000 7f0f77fff700 1 -- 172.21.15.113:0/417409240 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f7819ebe0 con 0x7f0f78103240 2024-05-03T18:38:30.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:30.029+0000 7f0f77fff700 1 --2- 172.21.15.113:0/417409240 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0f78103240 0x7f0f7819e6a0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f0f680056d0 tx=0x7f0f680057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:38:30.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:30.029+0000 7f0f757fa700 1 -- 172.21.15.113:0/417409240 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0f68026070 con 0x7f0f78103240 2024-05-03T18:38:30.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:30.029+0000 7f0f7e65e700 1 -- 172.21.15.113:0/417409240 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f7819eed0 con 0x7f0f78103240 2024-05-03T18:38:30.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:30.029+0000 7f0f757fa700 1 -- 172.21.15.113:0/417409240 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f6800bba0 con 0x7f0f78103240 2024-05-03T18:38:30.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:30.029+0000 7f0f757fa700 1 -- 172.21.15.113:0/417409240 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0f68003920 con 0x7f0f78103240 2024-05-03T18:38:30.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:30.029+0000 7f0f7e65e700 1 -- 172.21.15.113:0/417409240 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f78198820 con 0x7f0f78103240 2024-05-03T18:38:30.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:30.030+0000 7f0f757fa700 1 -- 172.21.15.113:0/417409240 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f0f68029070 con 0x7f0f78103240 2024-05-03T18:38:30.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:30.030+0000 7f0f7e65e700 1 -- 172.21.15.113:0/417409240 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f78104720 con 0x7f0f78103240 2024-05-03T18:38:30.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:30.031+0000 7f0f757fa700 1 --2- 172.21.15.113:0/417409240 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f0f6003e570 0x7f0f60040a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:38:30.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:30.031+0000 7f0f757fa700 1 -- 172.21.15.113:0/417409240 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f0f6801e070 con 0x7f0f78103240 2024-05-03T18:38:30.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:30.031+0000 7f0f777fe700 1 --2- 172.21.15.113:0/417409240 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f0f6003e570 0x7f0f60040a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:38:30.238 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:30.031+0000 7f0f777fe700 1 --2- 172.21.15.113:0/417409240 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f0f6003e570 0x7f0f60040a30 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f0f6c006fd0 tx=0x7f0f6c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:38:30.239 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:30.036+0000 7f0f757fa700 1 -- 172.21.15.113:0/417409240 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f0f68013070 con 0x7f0f78103240 2024-05-03T18:38:30.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:30.332+0000 7f0f7e65e700 1 -- 172.21.15.113:0/417409240 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f0f78199d50 con 0x7f0f78103240 2024-05-03T18:38:30.333 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:30.332+0000 7f0f757fa700 1 -- 172.21.15.113:0/417409240 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7f0f68030020 con 0x7f0f78103240 2024-05-03T18:38:30.333 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-05-03T18:38:30.333 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":1,"fsid":"f500455e-097b-11ef-bc97-c7b262605968","modified":"2024-05-03T18:36:40.734476Z","created":"2024-05-03T18:36:40.734476Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:3300","nonce":0},{"type":"v1","addr":"172.21.15.32:6789","nonce":0}]},"addr":"172.21.15.32:6789/0","public_addr":"172.21.15.32:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-03T18:38:30.335 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:30.335+0000 7f0f7e65e700 1 -- 172.21.15.113:0/417409240 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f0f6003e570 msgr2=0x7f0f60040a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:38:30.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:30.335+0000 7f0f7e65e700 1 --2- 172.21.15.113:0/417409240 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f0f6003e570 0x7f0f60040a30 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f0f6c006fd0 tx=0x7f0f6c006e40 comp rx=0 tx=0).stop 2024-05-03T18:38:30.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:30.335+0000 7f0f7e65e700 1 -- 172.21.15.113:0/417409240 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0f78103240 msgr2=0x7f0f7819e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:38:30.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:30.335+0000 7f0f7e65e700 1 --2- 172.21.15.113:0/417409240 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0f78103240 0x7f0f7819e6a0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f0f680056d0 tx=0x7f0f680057b0 comp rx=0 tx=0).stop 2024-05-03T18:38:30.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:30.335+0000 7f0f7e65e700 1 -- 172.21.15.113:0/417409240 shutdown_connections 2024-05-03T18:38:30.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:30.335+0000 7f0f7e65e700 1 --2- 172.21.15.113:0/417409240 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f0f6003e570 0x7f0f60040a30 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:30.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:30.335+0000 7f0f7e65e700 1 --2- 172.21.15.113:0/417409240 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0f78103240 0x7f0f7819e6a0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:30.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:30.335+0000 7f0f7e65e700 1 -- 172.21.15.113:0/417409240 >> 172.21.15.113:0/417409240 conn(0x7f0f780fea90 msgr2=0x7f0f78104c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:38:30.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:30.335+0000 7f0f7e65e700 1 -- 172.21.15.113:0/417409240 shutdown_connections 2024-05-03T18:38:30.336 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:30.335+0000 7f0f7e65e700 1 -- 172.21.15.113:0/417409240 wait complete. 2024-05-03T18:38:30.337 INFO:teuthology.orchestra.run.smithi113.stderr:dumped monmap epoch 1 2024-05-03T18:38:31.893 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-03T18:38:31.893 DEBUG:teuthology.orchestra.run.smithi113:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f500455e-097b-11ef-bc97-c7b262605968 -- ceph mon dump -f json 2024-05-03T18:38:32.137 INFO:teuthology.orchestra.run.smithi113.stderr:Inferring config /var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/mon.c/config 2024-05-03T18:38:35.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:38:35 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:38:35.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:35 smithi032 ceph-mon[29763]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:38:35.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:35 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-03T18:38:35.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:35 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-03T18:38:35.819 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:35 smithi032 ceph-mon[29763]: mon.a calling monitor election 2024-05-03T18:38:35.819 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:35 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-03T18:38:35.819 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:35 smithi032 ceph-mon[29763]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:38:35.819 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:35 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-03T18:38:35.819 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:35 smithi032 ceph-mon[29763]: mon.c calling monitor election 2024-05-03T18:38:35.819 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:35 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-03T18:38:35.819 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:35 smithi032 ceph-mon[29763]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:38:35.819 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:35 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-03T18:38:35.819 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:35 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-03T18:38:35.819 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:35 smithi032 ceph-mon[29763]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-03T18:38:35.819 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:35 smithi032 ceph-mon[29763]: monmap epoch 2 2024-05-03T18:38:35.819 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:35 smithi032 ceph-mon[29763]: fsid f500455e-097b-11ef-bc97-c7b262605968 2024-05-03T18:38:35.820 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:35 smithi032 ceph-mon[29763]: last_changed 2024-05-03T18:38:30.365439+0000 2024-05-03T18:38:35.820 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:35 smithi032 ceph-mon[29763]: created 2024-05-03T18:36:40.734476+0000 2024-05-03T18:38:35.820 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:35 smithi032 ceph-mon[29763]: min_mon_release 19 (squid) 2024-05-03T18:38:35.820 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:35 smithi032 ceph-mon[29763]: election_strategy: 1 2024-05-03T18:38:35.820 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:35 smithi032 ceph-mon[29763]: 0: [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] mon.a 2024-05-03T18:38:35.820 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:35 smithi032 ceph-mon[29763]: 1: [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] mon.c 2024-05-03T18:38:35.820 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:35 smithi032 ceph-mon[29763]: fsmap 2024-05-03T18:38:35.820 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:35 smithi032 ceph-mon[29763]: osdmap e4: 0 total, 0 up, 0 in 2024-05-03T18:38:35.820 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:35 smithi032 ceph-mon[29763]: mgrmap e13: x(active, since 49s) 2024-05-03T18:38:35.820 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:35 smithi032 ceph-mon[29763]: overall HEALTH_OK 2024-05-03T18:38:35.820 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:35 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:35.820 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:35 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:35.820 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:35 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:35.820 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:35 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-03T18:38:35.820 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:35 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:38:36.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:36.450+0000 7ff6a5fe6700 1 -- 172.21.15.113:0/3734189794 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6a00757c0 msgr2=0x7ff68c005610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:38:36.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:36.450+0000 7ff6a5fe6700 1 --2- 172.21.15.113:0/3734189794 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6a00757c0 0x7ff68c005610 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ff690009bf0 tx=0x7ff690009f00 comp rx=0 tx=0).stop 2024-05-03T18:38:36.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:36.450+0000 7ff6a5fe6700 1 -- 172.21.15.113:0/3734189794 shutdown_connections 2024-05-03T18:38:36.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:36.450+0000 7ff6a5fe6700 1 --2- 172.21.15.113:0/3734189794 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6a00757c0 0x7ff68c005610 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:36.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:36.450+0000 7ff6a5fe6700 1 -- 172.21.15.113:0/3734189794 >> 172.21.15.113:0/3734189794 conn(0x7ff6a00feab0 msgr2=0x7ff6a0100ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:38:36.451 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:36.450+0000 7ff6a5fe6700 1 -- 172.21.15.113:0/3734189794 shutdown_connections 2024-05-03T18:38:36.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:36.450+0000 7ff6a5fe6700 1 -- 172.21.15.113:0/3734189794 wait complete. 2024-05-03T18:38:36.452 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:36.451+0000 7ff6a5fe6700 1 Processor -- start 2024-05-03T18:38:36.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:36.451+0000 7ff6a5fe6700 1 -- start start 2024-05-03T18:38:36.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:36.451+0000 7ff6a5fe6700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff6a00757c0 0x7ff6a01a0840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:38:36.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:36.451+0000 7ff6a5fe6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6a01a0d80 0x7ff6a019aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:38:36.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:36.451+0000 7ff6a5fe6700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff6a010d120 con 0x7ff6a01a0d80 2024-05-03T18:38:36.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:36.451+0000 7ff6a5fe6700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7ff6a010cfa0 con 0x7ff6a00757c0 2024-05-03T18:38:36.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:36.451+0000 7ff69f7fe700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff6a00757c0 0x7ff6a01a0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:38:36.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:36.451+0000 7ff69f7fe700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff6a00757c0 0x7ff6a01a0840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:47646/0 (socket says 172.21.15.113:47646) 2024-05-03T18:38:36.454 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:36.451+0000 7ff69f7fe700 1 -- 172.21.15.113:0/4126527518 learned_addr learned my addr 172.21.15.113:0/4126527518 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-05-03T18:38:36.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:36.451+0000 7ff69effd700 1 --2- 172.21.15.113:0/4126527518 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6a01a0d80 0x7ff6a019aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:38:36.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:36.451+0000 7ff69f7fe700 1 -- 172.21.15.113:0/4126527518 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff6a00757c0 msgr2=0x7ff6a01a0840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-05-03T18:38:36.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:36.451+0000 7ff69f7fe700 1 -- 172.21.15.113:0/4126527518 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff6a00757c0 msgr2=0x7ff6a01a0840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-03T18:38:36.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:36.451+0000 7ff69f7fe700 1 --2- 172.21.15.113:0/4126527518 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff6a00757c0 0x7ff6a01a0840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-03T18:38:36.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:36.452+0000 7ff69f7fe700 1 --2- 172.21.15.113:0/4126527518 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff6a00757c0 0x7ff6a01a0840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-03T18:38:36.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:36.452+0000 7ff69effd700 1 -- 172.21.15.113:0/4126527518 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff6a00757c0 msgr2=0x7ff6a01a0840 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-03T18:38:36.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:36.452+0000 7ff69effd700 1 --2- 172.21.15.113:0/4126527518 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff6a00757c0 0x7ff6a01a0840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:36.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:36.452+0000 7ff69effd700 1 -- 172.21.15.113:0/4126527518 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6a01a13c0 con 0x7ff6a01a0d80 2024-05-03T18:38:36.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:36.452+0000 7ff69effd700 1 --2- 172.21.15.113:0/4126527518 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6a01a0d80 0x7ff6a019aa00 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff68800b4d0 tx=0x7ff68800b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:38:36.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:36.452+0000 7ff69cff9700 1 -- 172.21.15.113:0/4126527518 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7ff688011630 con 0x7ff6a01a0d80 2024-05-03T18:38:36.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:36.452+0000 7ff69cff9700 1 -- 172.21.15.113:0/4126527518 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff688011c80 con 0x7ff6a01a0d80 2024-05-03T18:38:36.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:36.452+0000 7ff6a5fe6700 1 -- 172.21.15.113:0/4126527518 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6a019af40 con 0x7ff6a01a0d80 2024-05-03T18:38:36.455 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:36.452+0000 7ff69cff9700 1 -- 172.21.15.113:0/4126527518 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7ff688010da0 con 0x7ff6a01a0d80 2024-05-03T18:38:36.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:36.452+0000 7ff6a5fe6700 1 -- 172.21.15.113:0/4126527518 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6a019b4d0 con 0x7ff6a01a0d80 2024-05-03T18:38:36.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:36.453+0000 7ff69cff9700 1 -- 172.21.15.113:0/4126527518 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7ff688018020 con 0x7ff6a01a0d80 2024-05-03T18:38:36.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:36.453+0000 7ff6a5fe6700 1 -- 172.21.15.113:0/4126527518 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6a0076f00 con 0x7ff6a01a0d80 2024-05-03T18:38:36.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:36.453+0000 7ff69cff9700 1 --2- 172.21.15.113:0/4126527518 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7ff68003e300 0x7ff6800407c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:38:36.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:36.454+0000 7ff69cff9700 1 -- 172.21.15.113:0/4126527518 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7ff688052130 con 0x7ff6a01a0d80 2024-05-03T18:38:36.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:36.454+0000 7ff69f7fe700 1 --2- 172.21.15.113:0/4126527518 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7ff68003e300 0x7ff6800407c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:38:36.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:36.454+0000 7ff69f7fe700 1 --2- 172.21.15.113:0/4126527518 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7ff68003e300 0x7ff6800407c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ff690006130 tx=0x7ff6900060c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:38:36.459 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:36.459+0000 7ff69cff9700 1 -- 172.21.15.113:0/4126527518 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7ff68801f020 con 0x7ff6a01a0d80 2024-05-03T18:38:36.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:36.752+0000 7ff6a5fe6700 1 -- 172.21.15.113:0/4126527518 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7ff6a019bf30 con 0x7ff6a01a0d80 2024-05-03T18:38:36.753 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:38:36 smithi113 ceph-mon[33540]: Deploying daemon mon.b on smithi045 2024-05-03T18:38:36.753 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:38:36 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-03T18:38:36.753 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:36.753+0000 7ff69cff9700 1 -- 172.21.15.113:0/4126527518 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7ff6880187b0 con 0x7ff6a01a0d80 2024-05-03T18:38:36.754 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-05-03T18:38:36.754 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":2,"fsid":"f500455e-097b-11ef-bc97-c7b262605968","modified":"2024-05-03T18:38:30.365439Z","created":"2024-05-03T18:36:40.734476Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:3300","nonce":0},{"type":"v1","addr":"172.21.15.32:6789","nonce":0}]},"addr":"172.21.15.32:6789/0","public_addr":"172.21.15.32:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:3300","nonce":0},{"type":"v1","addr":"172.21.15.113:6789","nonce":0}]},"addr":"172.21.15.113:6789/0","public_addr":"172.21.15.113:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-05-03T18:38:36.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:36.755+0000 7ff6a5fe6700 1 -- 172.21.15.113:0/4126527518 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7ff68003e300 msgr2=0x7ff6800407c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:38:36.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:36.755+0000 7ff6a5fe6700 1 --2- 172.21.15.113:0/4126527518 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7ff68003e300 0x7ff6800407c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ff690006130 tx=0x7ff6900060c0 comp rx=0 tx=0).stop 2024-05-03T18:38:36.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:36.755+0000 7ff6a5fe6700 1 -- 172.21.15.113:0/4126527518 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6a01a0d80 msgr2=0x7ff6a019aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:38:36.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:36.755+0000 7ff6a5fe6700 1 --2- 172.21.15.113:0/4126527518 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6a01a0d80 0x7ff6a019aa00 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff68800b4d0 tx=0x7ff68800b890 comp rx=0 tx=0).stop 2024-05-03T18:38:36.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:36.755+0000 7ff6a5fe6700 1 -- 172.21.15.113:0/4126527518 shutdown_connections 2024-05-03T18:38:36.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:36.755+0000 7ff6a5fe6700 1 --2- 172.21.15.113:0/4126527518 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7ff68003e300 0x7ff6800407c0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:36.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:36.755+0000 7ff6a5fe6700 1 --2- 172.21.15.113:0/4126527518 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff6a00757c0 0x7ff6a01a0840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:36.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:36.755+0000 7ff6a5fe6700 1 --2- 172.21.15.113:0/4126527518 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6a01a0d80 0x7ff6a019aa00 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:36.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:36.755+0000 7ff6a5fe6700 1 -- 172.21.15.113:0/4126527518 >> 172.21.15.113:0/4126527518 conn(0x7ff6a00feab0 msgr2=0x7ff6a0100980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:38:36.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:36.755+0000 7ff6a5fe6700 1 -- 172.21.15.113:0/4126527518 shutdown_connections 2024-05-03T18:38:36.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:36.755+0000 7ff6a5fe6700 1 -- 172.21.15.113:0/4126527518 wait complete. 2024-05-03T18:38:36.756 INFO:teuthology.orchestra.run.smithi113.stderr:dumped monmap epoch 2 2024-05-03T18:38:36.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:36 smithi032 ceph-mon[29763]: Deploying daemon mon.b on smithi045 2024-05-03T18:38:36.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:36 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-03T18:38:37.800 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:38:37 smithi113 ceph-mon[33540]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:38:37.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:38:37 smithi113 ceph-mon[33540]: from='client.? 172.21.15.113:0/4126527518' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-03T18:38:37.817 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:37 smithi032 ceph-mon[29763]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:38:37.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:37 smithi032 ceph-mon[29763]: from='client.? 172.21.15.113:0/4126527518' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-03T18:38:38.322 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-03T18:38:38.322 DEBUG:teuthology.orchestra.run.smithi113:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f500455e-097b-11ef-bc97-c7b262605968 -- ceph mon dump -f json 2024-05-03T18:38:38.566 INFO:teuthology.orchestra.run.smithi113.stderr:Inferring config /var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/mon.c/config 2024-05-03T18:38:39.121 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 podman[33279]: 2024-05-03 18:38:38.919433329 +0000 UTC m=+0.024447454 image pull ee4d846202496ba6096593e8e8becc0700a5eb7ebd95536031187f5f337d4012 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd 2024-05-03T18:38:39.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 podman[33279]: 2024-05-03 18:38:39.186892426 +0000 UTC m=+0.291906534 container init 8d050076af5619b684ab893d24573c7cb63577bae88639decd2c9150867584ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-mon-b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240430, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe) 2024-05-03T18:38:39.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 podman[33279]: 2024-05-03 18:38:39.189959862 +0000 UTC m=+0.294973965 container start 8d050076af5619b684ab893d24573c7cb63577bae88639decd2c9150867584ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-mon-b, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.build-date=20240430, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-05-03T18:38:39.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: set uid:gid to 167:167 (ceph:ceph) 2024-05-03T18:38:39.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: ceph version 19.0.0-3450-g9a047fe7 (9a047fe7599a549042d06a3fb5b6813a203eabea) squid (dev), process ceph-mon, pid 2 2024-05-03T18:38:39.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: pidfile_write: ignore empty --pid-file 2024-05-03T18:38:39.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: load: jerasure load: lrc 2024-05-03T18:38:39.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: RocksDB version: 7.9.2 2024-05-03T18:38:39.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Git sha 0 2024-05-03T18:38:39.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Compile date 2024-05-01 21:12:03 2024-05-03T18:38:39.499 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: DB SUMMARY 2024-05-03T18:38:39.499 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: DB Session ID: JM8SY9G9YFOUZZ8R1UQZ 2024-05-03T18:38:39.499 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: CURRENT file: CURRENT 2024-05-03T18:38:39.499 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: IDENTITY file: IDENTITY 2024-05-03T18:38:39.499 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-05-03T18:38:39.499 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-05-03T18:38:39.499 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2024-05-03T18:38:39.499 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.error_if_exists: 0 2024-05-03T18:38:39.499 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.create_if_missing: 0 2024-05-03T18:38:39.499 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.paranoid_checks: 1 2024-05-03T18:38:39.499 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.flush_verify_memtable_count: 1 2024-05-03T18:38:39.499 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-05-03T18:38:39.499 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-05-03T18:38:39.499 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.env: 0x5612bd3d6cc0 2024-05-03T18:38:39.500 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.fs: PosixFileSystem 2024-05-03T18:38:39.500 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.info_log: 0x5612be88dda0 2024-05-03T18:38:39.500 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.max_file_opening_threads: 16 2024-05-03T18:38:39.500 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.statistics: (nil) 2024-05-03T18:38:39.500 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.use_fsync: 0 2024-05-03T18:38:39.500 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.max_log_file_size: 0 2024-05-03T18:38:39.500 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-05-03T18:38:39.500 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.log_file_time_to_roll: 0 2024-05-03T18:38:39.500 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.keep_log_file_num: 1000 2024-05-03T18:38:39.500 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.recycle_log_file_num: 0 2024-05-03T18:38:39.500 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.allow_fallocate: 1 2024-05-03T18:38:39.500 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.allow_mmap_reads: 0 2024-05-03T18:38:39.500 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.allow_mmap_writes: 0 2024-05-03T18:38:39.500 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.use_direct_reads: 0 2024-05-03T18:38:39.501 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-05-03T18:38:39.501 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.create_missing_column_families: 0 2024-05-03T18:38:39.501 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.db_log_dir: 2024-05-03T18:38:39.501 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.wal_dir: 2024-05-03T18:38:39.501 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.table_cache_numshardbits: 6 2024-05-03T18:38:39.501 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.WAL_ttl_seconds: 0 2024-05-03T18:38:39.501 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.WAL_size_limit_MB: 0 2024-05-03T18:38:39.501 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-05-03T18:38:39.501 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-05-03T18:38:39.501 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.is_fd_close_on_exec: 1 2024-05-03T18:38:39.501 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.advise_random_on_open: 1 2024-05-03T18:38:39.501 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.db_write_buffer_size: 0 2024-05-03T18:38:39.501 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.write_buffer_manager: 0x5612be85f0e0 2024-05-03T18:38:39.501 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-05-03T18:38:39.501 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-05-03T18:38:39.502 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.use_adaptive_mutex: 0 2024-05-03T18:38:39.502 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.rate_limiter: (nil) 2024-05-03T18:38:39.502 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-05-03T18:38:39.502 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.wal_recovery_mode: 2 2024-05-03T18:38:39.502 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.enable_thread_tracking: 0 2024-05-03T18:38:39.502 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.enable_pipelined_write: 0 2024-05-03T18:38:39.502 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.unordered_write: 0 2024-05-03T18:38:39.502 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-05-03T18:38:39.502 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-05-03T18:38:39.502 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-05-03T18:38:39.502 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-05-03T18:38:39.502 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.row_cache: None 2024-05-03T18:38:39.502 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.wal_filter: None 2024-05-03T18:38:39.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-05-03T18:38:39.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.allow_ingest_behind: 0 2024-05-03T18:38:39.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.two_write_queues: 0 2024-05-03T18:38:39.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.manual_wal_flush: 0 2024-05-03T18:38:39.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.wal_compression: 0 2024-05-03T18:38:39.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.atomic_flush: 0 2024-05-03T18:38:39.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-05-03T18:38:39.505 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.persist_stats_to_disk: 0 2024-05-03T18:38:39.505 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.write_dbid_to_manifest: 0 2024-05-03T18:38:39.505 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.log_readahead_size: 0 2024-05-03T18:38:39.505 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-05-03T18:38:39.505 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.best_efforts_recovery: 0 2024-05-03T18:38:39.505 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-05-03T18:38:39.505 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-05-03T18:38:39.505 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.allow_data_in_errors: 0 2024-05-03T18:38:39.505 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.db_host_id: __hostname__ 2024-05-03T18:38:39.505 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.enforce_single_del_contracts: true 2024-05-03T18:38:39.505 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.max_background_jobs: 2 2024-05-03T18:38:39.505 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.max_background_compactions: -1 2024-05-03T18:38:39.505 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.max_subcompactions: 1 2024-05-03T18:38:39.505 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-05-03T18:38:39.506 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-05-03T18:38:39.506 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.delayed_write_rate : 16777216 2024-05-03T18:38:39.506 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.max_total_wal_size: 0 2024-05-03T18:38:39.506 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-05-03T18:38:39.506 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.stats_dump_period_sec: 600 2024-05-03T18:38:39.506 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.stats_persist_period_sec: 600 2024-05-03T18:38:39.506 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-05-03T18:38:39.506 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.max_open_files: -1 2024-05-03T18:38:39.506 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.bytes_per_sync: 0 2024-05-03T18:38:39.506 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.wal_bytes_per_sync: 0 2024-05-03T18:38:39.506 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.strict_bytes_per_sync: 0 2024-05-03T18:38:39.506 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.compaction_readahead_size: 0 2024-05-03T18:38:39.506 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.max_background_flushes: -1 2024-05-03T18:38:39.507 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Compression algorithms supported: 2024-05-03T18:38:39.507 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-05-03T18:38:39.507 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: kZSTD supported: 0 2024-05-03T18:38:39.507 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: kXpressCompression supported: 0 2024-05-03T18:38:39.507 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: kLZ4HCCompression supported: 1 2024-05-03T18:38:39.507 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: kZlibCompression supported: 1 2024-05-03T18:38:39.507 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: kSnappyCompression supported: 1 2024-05-03T18:38:39.507 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: kLZ4Compression supported: 1 2024-05-03T18:38:39.507 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: kBZip2Compression supported: 0 2024-05-03T18:38:39.507 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Fast CRC32 supported: Supported on x86 2024-05-03T18:38:39.507 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: DMutex implementation: pthread_mutex_t 2024-05-03T18:38:39.507 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-05-03T18:38:39.507 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-05-03T18:38:39.508 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-05-03T18:38:39.508 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.merge_operator: 2024-05-03T18:38:39.508 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.compaction_filter: None 2024-05-03T18:38:39.508 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.compaction_filter_factory: None 2024-05-03T18:38:39.508 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.sst_partitioner_factory: None 2024-05-03T18:38:39.508 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.memtable_factory: SkipListFactory 2024-05-03T18:38:39.508 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.table_factory: BlockBasedTable 2024-05-03T18:38:39.508 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5612be88df60) 2024-05-03T18:38:39.508 INFO:journalctl@ceph.mon.b.smithi045.stdout: cache_index_and_filter_blocks: 1 2024-05-03T18:38:39.508 INFO:journalctl@ceph.mon.b.smithi045.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-05-03T18:38:39.508 INFO:journalctl@ceph.mon.b.smithi045.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-05-03T18:38:39.508 INFO:journalctl@ceph.mon.b.smithi045.stdout: pin_top_level_index_and_filter: 1 2024-05-03T18:38:39.508 INFO:journalctl@ceph.mon.b.smithi045.stdout: index_type: 0 2024-05-03T18:38:39.508 INFO:journalctl@ceph.mon.b.smithi045.stdout: data_block_index_type: 0 2024-05-03T18:38:39.508 INFO:journalctl@ceph.mon.b.smithi045.stdout: index_shortening: 1 2024-05-03T18:38:39.509 INFO:journalctl@ceph.mon.b.smithi045.stdout: data_block_hash_table_util_ratio: 0.750000 2024-05-03T18:38:39.509 INFO:journalctl@ceph.mon.b.smithi045.stdout: checksum: 4 2024-05-03T18:38:39.509 INFO:journalctl@ceph.mon.b.smithi045.stdout: no_block_cache: 0 2024-05-03T18:38:39.509 INFO:journalctl@ceph.mon.b.smithi045.stdout: block_cache: 0x5612be8911f0 2024-05-03T18:38:39.509 INFO:journalctl@ceph.mon.b.smithi045.stdout: block_cache_name: BinnedLRUCache 2024-05-03T18:38:39.509 INFO:journalctl@ceph.mon.b.smithi045.stdout: block_cache_options: 2024-05-03T18:38:39.509 INFO:journalctl@ceph.mon.b.smithi045.stdout: capacity : 536870912 2024-05-03T18:38:39.509 INFO:journalctl@ceph.mon.b.smithi045.stdout: num_shard_bits : 4 2024-05-03T18:38:39.509 INFO:journalctl@ceph.mon.b.smithi045.stdout: strict_capacity_limit : 0 2024-05-03T18:38:39.509 INFO:journalctl@ceph.mon.b.smithi045.stdout: high_pri_pool_ratio: 0.000 2024-05-03T18:38:39.509 INFO:journalctl@ceph.mon.b.smithi045.stdout: block_cache_compressed: (nil) 2024-05-03T18:38:39.509 INFO:journalctl@ceph.mon.b.smithi045.stdout: persistent_cache: (nil) 2024-05-03T18:38:39.509 INFO:journalctl@ceph.mon.b.smithi045.stdout: block_size: 4096 2024-05-03T18:38:39.509 INFO:journalctl@ceph.mon.b.smithi045.stdout: block_size_deviation: 10 2024-05-03T18:38:39.509 INFO:journalctl@ceph.mon.b.smithi045.stdout: block_restart_interval: 16 2024-05-03T18:38:39.510 INFO:journalctl@ceph.mon.b.smithi045.stdout: index_block_restart_interval: 1 2024-05-03T18:38:39.510 INFO:journalctl@ceph.mon.b.smithi045.stdout: metadata_block_size: 4096 2024-05-03T18:38:39.510 INFO:journalctl@ceph.mon.b.smithi045.stdout: partition_filters: 0 2024-05-03T18:38:39.510 INFO:journalctl@ceph.mon.b.smithi045.stdout: use_delta_encoding: 1 2024-05-03T18:38:39.510 INFO:journalctl@ceph.mon.b.smithi045.stdout: filter_policy: bloomfilter 2024-05-03T18:38:39.510 INFO:journalctl@ceph.mon.b.smithi045.stdout: whole_key_filtering: 1 2024-05-03T18:38:39.510 INFO:journalctl@ceph.mon.b.smithi045.stdout: verify_compression: 0 2024-05-03T18:38:39.510 INFO:journalctl@ceph.mon.b.smithi045.stdout: read_amp_bytes_per_bit: 0 2024-05-03T18:38:39.510 INFO:journalctl@ceph.mon.b.smithi045.stdout: format_version: 5 2024-05-03T18:38:39.510 INFO:journalctl@ceph.mon.b.smithi045.stdout: enable_index_compression: 1 2024-05-03T18:38:39.510 INFO:journalctl@ceph.mon.b.smithi045.stdout: block_align: 0 2024-05-03T18:38:39.510 INFO:journalctl@ceph.mon.b.smithi045.stdout: max_auto_readahead_size: 262144 2024-05-03T18:38:39.510 INFO:journalctl@ceph.mon.b.smithi045.stdout: prepopulate_block_cache: 0 2024-05-03T18:38:39.510 INFO:journalctl@ceph.mon.b.smithi045.stdout: initial_auto_readahead_size: 8192 2024-05-03T18:38:39.510 INFO:journalctl@ceph.mon.b.smithi045.stdout: num_file_reads_for_auto_readahead: 2 2024-05-03T18:38:39.511 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.write_buffer_size: 33554432 2024-05-03T18:38:39.511 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.max_write_buffer_number: 2 2024-05-03T18:38:39.511 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.compression: NoCompression 2024-05-03T18:38:39.511 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.bottommost_compression: Disabled 2024-05-03T18:38:39.511 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.prefix_extractor: nullptr 2024-05-03T18:38:39.511 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-05-03T18:38:39.511 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.num_levels: 7 2024-05-03T18:38:39.511 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-05-03T18:38:39.511 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-05-03T18:38:39.511 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-05-03T18:38:39.511 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-05-03T18:38:39.511 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-05-03T18:38:39.511 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-05-03T18:38:39.511 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-05-03T18:38:39.511 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-05-03T18:38:39.512 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-05-03T18:38:39.512 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-05-03T18:38:39.512 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-05-03T18:38:39.512 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-05-03T18:38:39.512 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.compression_opts.window_bits: -14 2024-05-03T18:38:39.512 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.compression_opts.level: 32767 2024-05-03T18:38:39.512 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.compression_opts.strategy: 0 2024-05-03T18:38:39.512 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-05-03T18:38:39.512 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-05-03T18:38:39.512 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-05-03T18:38:39.512 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-05-03T18:38:39.512 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.compression_opts.enabled: false 2024-05-03T18:38:39.512 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-05-03T18:38:39.512 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-05-03T18:38:39.512 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-05-03T18:38:39.513 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-05-03T18:38:39.513 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.target_file_size_base: 67108864 2024-05-03T18:38:39.513 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.target_file_size_multiplier: 1 2024-05-03T18:38:39.513 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-05-03T18:38:39.513 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-05-03T18:38:39.513 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-05-03T18:38:39.513 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-05-03T18:38:39.513 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-05-03T18:38:39.513 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-05-03T18:38:39.513 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-05-03T18:38:39.513 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-05-03T18:38:39.513 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-05-03T18:38:39.513 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-05-03T18:38:39.513 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-05-03T18:38:39.514 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-05-03T18:38:39.514 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-05-03T18:38:39.514 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.arena_block_size: 1048576 2024-05-03T18:38:39.514 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-05-03T18:38:39.514 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-05-03T18:38:39.514 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.disable_auto_compactions: 0 2024-05-03T18:38:39.514 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-05-03T18:38:39.514 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-05-03T18:38:39.514 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-05-03T18:38:39.514 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-05-03T18:38:39.514 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-05-03T18:38:39.514 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-05-03T18:38:39.514 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-05-03T18:38:39.514 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-05-03T18:38:39.515 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-05-03T18:38:39.515 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-05-03T18:38:39.515 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-05-03T18:38:39.515 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.inplace_update_support: 0 2024-05-03T18:38:39.515 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.inplace_update_num_locks: 10000 2024-05-03T18:38:39.515 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-05-03T18:38:39.515 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-05-03T18:38:39.515 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.memtable_huge_page_size: 0 2024-05-03T18:38:39.515 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.bloom_locality: 0 2024-05-03T18:38:39.515 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.max_successive_merges: 0 2024-05-03T18:38:39.515 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.optimize_filters_for_hits: 0 2024-05-03T18:38:39.515 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.paranoid_file_checks: 0 2024-05-03T18:38:39.515 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.force_consistency_checks: 1 2024-05-03T18:38:39.515 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.report_bg_io_stats: 0 2024-05-03T18:38:39.515 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.ttl: 2592000 2024-05-03T18:38:39.515 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.periodic_compaction_seconds: 0 2024-05-03T18:38:39.516 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-05-03T18:38:39.516 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-05-03T18:38:39.516 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.enable_blob_files: false 2024-05-03T18:38:39.516 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.min_blob_size: 0 2024-05-03T18:38:39.516 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.blob_file_size: 268435456 2024-05-03T18:38:39.516 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.blob_compression_type: NoCompression 2024-05-03T18:38:39.516 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.enable_blob_garbage_collection: false 2024-05-03T18:38:39.516 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-05-03T18:38:39.516 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-05-03T18:38:39.516 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-05-03T18:38:39.516 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.blob_file_starting_level: 0 2024-05-03T18:38:39.516 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-05-03T18:38:39.516 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-05-03T18:38:39.517 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-05-03T18:38:39.517 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: c54f1222-0475-4639-b1ca-b822150013a1 2024-05-03T18:38:39.517 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714761519221031, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-05-03T18:38:39.517 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-05-03T18:38:39.517 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714761519221631, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1773, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1714761519, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "c54f1222-0475-4639-b1ca-b822150013a1", "db_session_id": "JM8SY9G9YFOUZZ8R1UQZ", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-05-03T18:38:39.517 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714761519221742, "job": 1, "event": "recovery_finished"} 2024-05-03T18:38:39.517 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-05-03T18:38:39.517 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-03T18:38:39.517 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5612be9ae000 2024-05-03T18:38:39.517 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: DB pointer 0x5612be99a000 2024-05-03T18:38:39.517 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-05-03T18:38:39.517 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-05-03T18:38:39.517 INFO:journalctl@ceph.mon.b.smithi045.stdout: ** DB Stats ** 2024-05-03T18:38:39.517 INFO:journalctl@ceph.mon.b.smithi045.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-03T18:38:39.517 INFO:journalctl@ceph.mon.b.smithi045.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-03T18:38:39.518 INFO:journalctl@ceph.mon.b.smithi045.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-03T18:38:39.518 INFO:journalctl@ceph.mon.b.smithi045.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-03T18:38:39.518 INFO:journalctl@ceph.mon.b.smithi045.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-03T18:38:39.518 INFO:journalctl@ceph.mon.b.smithi045.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-03T18:38:39.518 INFO:journalctl@ceph.mon.b.smithi045.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-03T18:38:39.518 INFO:journalctl@ceph.mon.b.smithi045.stdout: 2024-05-03T18:38:39.518 INFO:journalctl@ceph.mon.b.smithi045.stdout: ** Compaction Stats [default] ** 2024-05-03T18:38:39.518 INFO:journalctl@ceph.mon.b.smithi045.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-03T18:38:39.518 INFO:journalctl@ceph.mon.b.smithi045.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-05-03T18:38:39.518 INFO:journalctl@ceph.mon.b.smithi045.stdout: L0 1/0 1.73 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-03T18:38:39.518 INFO:journalctl@ceph.mon.b.smithi045.stdout: Sum 1/0 1.73 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-03T18:38:39.518 INFO:journalctl@ceph.mon.b.smithi045.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-03T18:38:39.518 INFO:journalctl@ceph.mon.b.smithi045.stdout: 2024-05-03T18:38:39.518 INFO:journalctl@ceph.mon.b.smithi045.stdout: ** Compaction Stats [default] ** 2024-05-03T18:38:39.519 INFO:journalctl@ceph.mon.b.smithi045.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-03T18:38:39.519 INFO:journalctl@ceph.mon.b.smithi045.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-03T18:38:39.519 INFO:journalctl@ceph.mon.b.smithi045.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-03T18:38:39.519 INFO:journalctl@ceph.mon.b.smithi045.stdout: 2024-05-03T18:38:39.519 INFO:journalctl@ceph.mon.b.smithi045.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-05-03T18:38:39.519 INFO:journalctl@ceph.mon.b.smithi045.stdout: 2024-05-03T18:38:39.519 INFO:journalctl@ceph.mon.b.smithi045.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-03T18:38:39.519 INFO:journalctl@ceph.mon.b.smithi045.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-05-03T18:38:39.519 INFO:journalctl@ceph.mon.b.smithi045.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-03T18:38:39.519 INFO:journalctl@ceph.mon.b.smithi045.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-03T18:38:39.519 INFO:journalctl@ceph.mon.b.smithi045.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-03T18:38:39.519 INFO:journalctl@ceph.mon.b.smithi045.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-03T18:38:39.519 INFO:journalctl@ceph.mon.b.smithi045.stdout: Cumulative compaction: 0.00 GB write, 0.47 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-03T18:38:39.519 INFO:journalctl@ceph.mon.b.smithi045.stdout: Interval compaction: 0.00 GB write, 0.47 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-03T18:38:39.519 INFO:journalctl@ceph.mon.b.smithi045.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-03T18:38:39.520 INFO:journalctl@ceph.mon.b.smithi045.stdout: Block cache BinnedLRUCache@0x5612be8911f0#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-05-03T18:38:39.520 INFO:journalctl@ceph.mon.b.smithi045.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-05-03T18:38:39.520 INFO:journalctl@ceph.mon.b.smithi045.stdout: 2024-05-03T18:38:39.520 INFO:journalctl@ceph.mon.b.smithi045.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-03T18:38:39.520 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-05-03T18:38:39.520 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: using public_addr v2:172.21.15.45:0/0 -> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] 2024-05-03T18:38:39.520 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: starting mon.b rank -1 at public addrs [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] at bind addrs [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid f500455e-097b-11ef-bc97-c7b262605968 2024-05-03T18:38:39.520 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: mon.b@-1(???) e0 preinit fsid f500455e-097b-11ef-bc97-c7b262605968 2024-05-03T18:38:39.520 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: mon.b@-1(synchronizing).mds e1 new map 2024-05-03T18:38:39.520 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: mon.b@-1(synchronizing).mds e1 print_map 2024-05-03T18:38:39.520 INFO:journalctl@ceph.mon.b.smithi045.stdout: e1 2024-05-03T18:38:39.520 INFO:journalctl@ceph.mon.b.smithi045.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-03T18:38:39.520 INFO:journalctl@ceph.mon.b.smithi045.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-03T18:38:39.520 INFO:journalctl@ceph.mon.b.smithi045.stdout: legacy client fscid: -1 2024-05-03T18:38:39.520 INFO:journalctl@ceph.mon.b.smithi045.stdout: 2024-05-03T18:38:39.521 INFO:journalctl@ceph.mon.b.smithi045.stdout: No filesystems configured 2024-05-03T18:38:39.521 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-05-03T18:38:39.521 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-05-03T18:38:39.521 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-05-03T18:38:39.521 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-05-03T18:38:39.521 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-05-03T18:38:39.521 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-05-03T18:38:39.521 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-05-03T18:38:39.521 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-03T18:38:39.521 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-03T18:38:39.521 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-03T18:38:39.521 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:38:39.521 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:39.521 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: Added host smithi045 2024-05-03T18:38:39.522 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:38:39.522 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:39.522 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:38:39.522 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:39.522 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:38:39.522 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-03T18:38:39.522 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:39.522 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:38:39.522 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi113", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:38:39.522 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: Deploying cephadm binary to smithi113 2024-05-03T18:38:39.522 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:39.522 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:39.522 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:39.522 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:39.522 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi045", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:38:39.523 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:38:39.523 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:38:39.523 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: Updating smithi045:/etc/ceph/ceph.conf 2024-05-03T18:38:39.523 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: Updating smithi045:/var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-03T18:38:39.523 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:38:39.523 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:38:39.523 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: Updating smithi045:/etc/ceph/ceph.client.admin.keyring 2024-05-03T18:38:39.523 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: Updating smithi045:/var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-03T18:38:39.523 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:39.523 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:39.523 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:39.523 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:39.523 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:38:39.523 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:39.523 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: Added host smithi113 2024-05-03T18:38:39.524 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:39.524 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:38:39.524 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:39.524 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-03T18:38:39.524 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:38:39.524 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:39.524 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:39.524 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:39.524 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:39.524 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi113", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:38:39.524 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:38:39.524 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:38:39.524 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: Updating smithi113:/etc/ceph/ceph.conf 2024-05-03T18:38:39.524 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:38:39.525 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: from='client.? 172.21.15.32:0/566430011' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-05-03T18:38:39.525 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: Updating smithi113:/var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-03T18:38:39.525 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: Updating smithi113:/etc/ceph/ceph.client.admin.keyring 2024-05-03T18:38:39.525 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: Updating smithi113:/var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-03T18:38:39.525 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:39.525 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:39.525 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:39.525 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: from='client.? 172.21.15.32:0/566430011' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-05-03T18:38:39.525 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: osdmap e4: 0 total, 0 up, 0 in 2024-05-03T18:38:39.525 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:38:39.525 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi032:172.21.15.32=a;smithi045:172.21.15.45=b;smithi113:172.21.15.113=c", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:38:39.525 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: Saving service mon spec with placement smithi032:172.21.15.32=a;smithi045:172.21.15.45=b;smithi113:172.21.15.113=c;count:3 2024-05-03T18:38:39.525 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:39.525 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:38:39.525 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:38:39.526 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:38:39.526 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:39.526 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-03T18:38:39.526 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:38:39.526 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: Deploying daemon mon.c on smithi113 2024-05-03T18:38:39.526 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:38:39.526 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:38:39.526 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:38:39.526 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-03T18:38:39.526 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-03T18:38:39.526 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: mon.a calling monitor election 2024-05-03T18:38:39.526 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-03T18:38:39.526 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:38:39.526 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-03T18:38:39.527 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: mon.c calling monitor election 2024-05-03T18:38:39.527 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-03T18:38:39.527 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:38:39.527 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-03T18:38:39.527 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-03T18:38:39.527 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-03T18:38:39.527 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: monmap epoch 2 2024-05-03T18:38:39.527 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: fsid f500455e-097b-11ef-bc97-c7b262605968 2024-05-03T18:38:39.527 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: last_changed 2024-05-03T18:38:30.365439+0000 2024-05-03T18:38:39.527 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: created 2024-05-03T18:36:40.734476+0000 2024-05-03T18:38:39.527 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: min_mon_release 19 (squid) 2024-05-03T18:38:39.527 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: election_strategy: 1 2024-05-03T18:38:39.527 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: 0: [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] mon.a 2024-05-03T18:38:39.527 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: 1: [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] mon.c 2024-05-03T18:38:39.527 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: fsmap 2024-05-03T18:38:39.528 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: osdmap e4: 0 total, 0 up, 0 in 2024-05-03T18:38:39.528 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: mgrmap e13: x(active, since 49s) 2024-05-03T18:38:39.528 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: overall HEALTH_OK 2024-05-03T18:38:39.528 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:39.528 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:39.528 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:39.528 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-03T18:38:39.528 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:38:39.528 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: Deploying daemon mon.b on smithi045 2024-05-03T18:38:39.528 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-03T18:38:39.528 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:38:39.528 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: from='client.? 172.21.15.113:0/4126527518' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-03T18:38:39.528 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 ceph-mon[33299]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-05-03T18:38:39.528 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 bash[33279]: 8d050076af5619b684ab893d24573c7cb63577bae88639decd2c9150867584ed 2024-05-03T18:38:39.528 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:39 smithi045 systemd[1]: Started Ceph mon.b for f500455e-097b-11ef-bc97-c7b262605968. 2024-05-03T18:38:44.354 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:44.352+0000 7fa63662e700 1 -- 172.21.15.113:0/3077103082 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa6180035c0 msgr2=0x7fa618005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:38:44.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:44.352+0000 7fa63662e700 1 --2- 172.21.15.113:0/3077103082 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa6180035c0 0x7fa618005a50 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa630102ef0 tx=0x7fa62000fe60 comp rx=0 tx=0).stop 2024-05-03T18:38:44.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:44.352+0000 7fa63662e700 1 -- 172.21.15.113:0/3077103082 shutdown_connections 2024-05-03T18:38:44.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:44.352+0000 7fa63662e700 1 --2- 172.21.15.113:0/3077103082 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa6180035c0 0x7fa618005a50 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:44.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:44.352+0000 7fa63662e700 1 -- 172.21.15.113:0/3077103082 >> 172.21.15.113:0/3077103082 conn(0x7fa6300feab0 msgr2=0x7fa630100ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:38:44.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:44.352+0000 7fa63662e700 1 -- 172.21.15.113:0/3077103082 shutdown_connections 2024-05-03T18:38:44.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:44.352+0000 7fa63662e700 1 -- 172.21.15.113:0/3077103082 wait complete. 2024-05-03T18:38:44.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:44.353+0000 7fa63662e700 1 Processor -- start 2024-05-03T18:38:44.355 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:44.353+0000 7fa63662e700 1 -- start start 2024-05-03T18:38:44.357 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:44.354+0000 7fa63662e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa6180035c0 0x7fa6300757a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:38:44.357 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:44.354+0000 7fa63662e700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa630075ce0 0x7fa630076160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:38:44.357 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:44.354+0000 7fa63662e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa6301a4df0 0x7fa6301a71e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:38:44.357 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:44.354+0000 7fa63662e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa63010aeb0 con 0x7fa6301a4df0 2024-05-03T18:38:44.357 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:44.354+0000 7fa63662e700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa63010ad30 con 0x7fa6180035c0 2024-05-03T18:38:44.357 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:44.354+0000 7fa63662e700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fa63010b030 con 0x7fa630075ce0 2024-05-03T18:38:44.357 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:44.354+0000 7fa62f7fe700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa630075ce0 0x7fa630076160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:38:44.357 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:44.354+0000 7fa62f7fe700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa630075ce0 0x7fa630076160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:40088/0 (socket says 172.21.15.113:40088) 2024-05-03T18:38:44.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:44.354+0000 7fa634bcb700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa6301a4df0 0x7fa6301a71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:38:44.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:44.354+0000 7fa62f7fe700 1 -- 172.21.15.113:0/2508921939 learned_addr learned my addr 172.21.15.113:0/2508921939 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-05-03T18:38:44.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:44.354+0000 7fa62ffff700 1 --2- 172.21.15.113:0/2508921939 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa6180035c0 0x7fa6300757a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:38:44.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:44.354+0000 7fa62f7fe700 1 -- 172.21.15.113:0/2508921939 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa630075ce0 msgr2=0x7fa630076160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-05-03T18:38:44.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:44.354+0000 7fa62f7fe700 1 -- 172.21.15.113:0/2508921939 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa630075ce0 msgr2=0x7fa630076160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-03T18:38:44.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:44.354+0000 7fa62f7fe700 1 --2- 172.21.15.113:0/2508921939 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa630075ce0 0x7fa630076160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-03T18:38:44.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:44.354+0000 7fa62f7fe700 1 --2- 172.21.15.113:0/2508921939 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa630075ce0 0x7fa630076160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-03T18:38:44.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:44.354+0000 7fa62ffff700 1 -- 172.21.15.113:0/2508921939 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa6180035c0 msgr2=0x7fa6300757a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-05-03T18:38:44.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:44.354+0000 7fa62ffff700 1 -- 172.21.15.113:0/2508921939 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa6180035c0 msgr2=0x7fa6300757a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-03T18:38:44.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:44.354+0000 7fa62ffff700 1 --2- 172.21.15.113:0/2508921939 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa6180035c0 0x7fa6300757a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-03T18:38:44.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:44.354+0000 7fa634bcb700 1 -- 172.21.15.113:0/2508921939 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa630075ce0 msgr2=0x7fa630076160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-03T18:38:44.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:44.354+0000 7fa634bcb700 1 --2- 172.21.15.113:0/2508921939 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa630075ce0 0x7fa630076160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:44.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:44.354+0000 7fa62ffff700 1 --2- 172.21.15.113:0/2508921939 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa6180035c0 0x7fa6300757a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-03T18:38:44.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:44.354+0000 7fa634bcb700 1 -- 172.21.15.113:0/2508921939 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa6180035c0 msgr2=0x7fa6300757a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-03T18:38:44.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:44.354+0000 7fa634bcb700 1 --2- 172.21.15.113:0/2508921939 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa6180035c0 0x7fa6300757a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:44.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:44.354+0000 7fa634bcb700 1 -- 172.21.15.113:0/2508921939 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6301a78b0 con 0x7fa6301a4df0 2024-05-03T18:38:44.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:44.354+0000 7fa634bcb700 1 --2- 172.21.15.113:0/2508921939 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa6301a4df0 0x7fa6301a71e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa62800bd50 tx=0x7fa62800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:38:44.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:44.355+0000 7fa62d7fa700 1 -- 172.21.15.113:0/2508921939 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa628019070 con 0x7fa6301a4df0 2024-05-03T18:38:44.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:44.355+0000 7fa62d7fa700 1 -- 172.21.15.113:0/2508921939 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa62800cd10 con 0x7fa6301a4df0 2024-05-03T18:38:44.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:44.355+0000 7fa63662e700 1 -- 172.21.15.113:0/2508921939 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6301a7ba0 con 0x7fa6301a4df0 2024-05-03T18:38:44.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:44.355+0000 7fa62d7fa700 1 -- 172.21.15.113:0/2508921939 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa628012700 con 0x7fa6301a4df0 2024-05-03T18:38:44.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:44.355+0000 7fa63662e700 1 -- 172.21.15.113:0/2508921939 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6301a80e0 con 0x7fa6301a4df0 2024-05-03T18:38:44.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:44.356+0000 7fa62d7fa700 1 -- 172.21.15.113:0/2508921939 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7fa62801c070 con 0x7fa6301a4df0 2024-05-03T18:38:44.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:44.356+0000 7fa62d7fa700 1 --2- 172.21.15.113:0/2508921939 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fa61c03e9a0 0x7fa61c040e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:38:44.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:44.356+0000 7fa62d7fa700 1 -- 172.21.15.113:0/2508921939 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fa628052080 con 0x7fa6301a4df0 2024-05-03T18:38:44.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:44.356+0000 7fa63662e700 1 -- 172.21.15.113:0/2508921939 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa630064b00 con 0x7fa6301a4df0 2024-05-03T18:38:44.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:44.356+0000 7fa62ffff700 1 --2- 172.21.15.113:0/2508921939 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fa61c03e9a0 0x7fa61c040e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:38:44.360 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:44.357+0000 7fa62ffff700 1 --2- 172.21.15.113:0/2508921939 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fa61c03e9a0 0x7fa61c040e60 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fa62000fa10 tx=0x7fa620005a90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:38:44.363 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:44.362+0000 7fa62d7fa700 1 -- 172.21.15.113:0/2508921939 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fa628017120 con 0x7fa6301a4df0 2024-05-03T18:38:44.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:44 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-03T18:38:44.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:44 smithi032 ceph-mon[29763]: mon.a calling monitor election 2024-05-03T18:38:44.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:44 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-03T18:38:44.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:44 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-03T18:38:44.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:44 smithi032 ceph-mon[29763]: mon.c calling monitor election 2024-05-03T18:38:44.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:44 smithi032 ceph-mon[29763]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:38:44.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:44 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-03T18:38:44.569 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:44 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-03T18:38:44.569 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:44 smithi032 ceph-mon[29763]: mon.b calling monitor election 2024-05-03T18:38:44.569 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:44 smithi032 ceph-mon[29763]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:38:44.569 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:44 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-03T18:38:44.569 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:44 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-03T18:38:44.569 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:44 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-03T18:38:44.569 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:44 smithi032 ceph-mon[29763]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-03T18:38:44.569 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:44 smithi032 ceph-mon[29763]: monmap epoch 3 2024-05-03T18:38:44.569 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:44 smithi032 ceph-mon[29763]: fsid f500455e-097b-11ef-bc97-c7b262605968 2024-05-03T18:38:44.569 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:44 smithi032 ceph-mon[29763]: last_changed 2024-05-03T18:38:39.248717+0000 2024-05-03T18:38:44.569 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:44 smithi032 ceph-mon[29763]: created 2024-05-03T18:36:40.734476+0000 2024-05-03T18:38:44.569 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:44 smithi032 ceph-mon[29763]: min_mon_release 19 (squid) 2024-05-03T18:38:44.569 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:44 smithi032 ceph-mon[29763]: election_strategy: 1 2024-05-03T18:38:44.569 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:44 smithi032 ceph-mon[29763]: 0: [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] mon.a 2024-05-03T18:38:44.570 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:44 smithi032 ceph-mon[29763]: 1: [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] mon.c 2024-05-03T18:38:44.570 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:44 smithi032 ceph-mon[29763]: 2: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2024-05-03T18:38:44.570 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:44 smithi032 ceph-mon[29763]: fsmap 2024-05-03T18:38:44.570 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:44 smithi032 ceph-mon[29763]: osdmap e4: 0 total, 0 up, 0 in 2024-05-03T18:38:44.570 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:44 smithi032 ceph-mon[29763]: mgrmap e13: x(active, since 58s) 2024-05-03T18:38:44.570 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:44 smithi032 ceph-mon[29763]: overall HEALTH_OK 2024-05-03T18:38:44.570 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:44 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:44.570 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:44 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:44.570 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:44 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:44.570 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:44 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:44.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:44.664+0000 7fa63662e700 1 -- 172.21.15.113:0/2508921939 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fa6301a8370 con 0x7fa6301a4df0 2024-05-03T18:38:44.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:44.665+0000 7fa62d7fa700 1 -- 172.21.15.113:0/2508921939 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1281 (secure 0 0 0) 0x7fa628012c40 con 0x7fa6301a4df0 2024-05-03T18:38:44.667 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-05-03T18:38:44.667 INFO:teuthology.orchestra.run.smithi113.stdout:{"epoch":3,"fsid":"f500455e-097b-11ef-bc97-c7b262605968","modified":"2024-05-03T18:38:39.248717Z","created":"2024-05-03T18:36:40.734476Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:3300","nonce":0},{"type":"v1","addr":"172.21.15.32:6789","nonce":0}]},"addr":"172.21.15.32:6789/0","public_addr":"172.21.15.32:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:3300","nonce":0},{"type":"v1","addr":"172.21.15.113:6789","nonce":0}]},"addr":"172.21.15.113:6789/0","public_addr":"172.21.15.113:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:3300","nonce":0},{"type":"v1","addr":"172.21.15.45:6789","nonce":0}]},"addr":"172.21.15.45:6789/0","public_addr":"172.21.15.45:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-05-03T18:38:44.667 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:38:44 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-03T18:38:44.667 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:38:44 smithi113 ceph-mon[33540]: mon.a calling monitor election 2024-05-03T18:38:44.667 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:38:44 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-03T18:38:44.667 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:38:44 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-03T18:38:44.667 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:38:44 smithi113 ceph-mon[33540]: mon.c calling monitor election 2024-05-03T18:38:44.668 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:38:44 smithi113 ceph-mon[33540]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:38:44.668 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:38:44 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-03T18:38:44.668 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:38:44 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-03T18:38:44.668 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:38:44 smithi113 ceph-mon[33540]: mon.b calling monitor election 2024-05-03T18:38:44.668 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:38:44 smithi113 ceph-mon[33540]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:38:44.668 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:38:44 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-03T18:38:44.668 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:38:44 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-03T18:38:44.668 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:38:44 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-03T18:38:44.668 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:38:44 smithi113 ceph-mon[33540]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-03T18:38:44.668 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:38:44 smithi113 ceph-mon[33540]: monmap epoch 3 2024-05-03T18:38:44.668 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:38:44 smithi113 ceph-mon[33540]: fsid f500455e-097b-11ef-bc97-c7b262605968 2024-05-03T18:38:44.668 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:38:44 smithi113 ceph-mon[33540]: last_changed 2024-05-03T18:38:39.248717+0000 2024-05-03T18:38:44.668 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:38:44 smithi113 ceph-mon[33540]: created 2024-05-03T18:36:40.734476+0000 2024-05-03T18:38:44.668 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:38:44 smithi113 ceph-mon[33540]: min_mon_release 19 (squid) 2024-05-03T18:38:44.668 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:38:44 smithi113 ceph-mon[33540]: election_strategy: 1 2024-05-03T18:38:44.668 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:38:44 smithi113 ceph-mon[33540]: 0: [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] mon.a 2024-05-03T18:38:44.669 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:38:44 smithi113 ceph-mon[33540]: 1: [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] mon.c 2024-05-03T18:38:44.669 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:38:44 smithi113 ceph-mon[33540]: 2: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2024-05-03T18:38:44.669 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:38:44 smithi113 ceph-mon[33540]: fsmap 2024-05-03T18:38:44.669 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:38:44 smithi113 ceph-mon[33540]: osdmap e4: 0 total, 0 up, 0 in 2024-05-03T18:38:44.669 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:38:44 smithi113 ceph-mon[33540]: mgrmap e13: x(active, since 58s) 2024-05-03T18:38:44.669 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:38:44 smithi113 ceph-mon[33540]: overall HEALTH_OK 2024-05-03T18:38:44.669 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:38:44 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:44.669 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:38:44 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:44.669 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:38:44 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:44.669 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:38:44 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:44.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:44.667+0000 7fa63662e700 1 -- 172.21.15.113:0/2508921939 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fa61c03e9a0 msgr2=0x7fa61c040e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:38:44.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:44.667+0000 7fa63662e700 1 --2- 172.21.15.113:0/2508921939 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fa61c03e9a0 0x7fa61c040e60 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fa62000fa10 tx=0x7fa620005a90 comp rx=0 tx=0).stop 2024-05-03T18:38:44.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:44.667+0000 7fa63662e700 1 -- 172.21.15.113:0/2508921939 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa6301a4df0 msgr2=0x7fa6301a71e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:38:44.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:44.667+0000 7fa63662e700 1 --2- 172.21.15.113:0/2508921939 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa6301a4df0 0x7fa6301a71e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa62800bd50 tx=0x7fa62800be30 comp rx=0 tx=0).stop 2024-05-03T18:38:44.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:44.667+0000 7fa63662e700 1 -- 172.21.15.113:0/2508921939 shutdown_connections 2024-05-03T18:38:44.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:44.667+0000 7fa63662e700 1 --2- 172.21.15.113:0/2508921939 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fa61c03e9a0 0x7fa61c040e60 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:44.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:44.667+0000 7fa63662e700 1 --2- 172.21.15.113:0/2508921939 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa6301a4df0 0x7fa6301a71e0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:44.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:44.667+0000 7fa63662e700 1 --2- 172.21.15.113:0/2508921939 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa6180035c0 0x7fa6300757a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:44.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:44.667+0000 7fa63662e700 1 --2- 172.21.15.113:0/2508921939 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa630075ce0 0x7fa630076160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:44.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:44.667+0000 7fa63662e700 1 -- 172.21.15.113:0/2508921939 >> 172.21.15.113:0/2508921939 conn(0x7fa6300feab0 msgr2=0x7fa630106c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:38:44.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:44.667+0000 7fa63662e700 1 -- 172.21.15.113:0/2508921939 shutdown_connections 2024-05-03T18:38:44.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:44.668+0000 7fa63662e700 1 -- 172.21.15.113:0/2508921939 wait complete. 2024-05-03T18:38:44.671 INFO:teuthology.orchestra.run.smithi113.stderr:dumped monmap epoch 3 2024-05-03T18:38:44.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:44 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-03T18:38:44.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:44 smithi045 ceph-mon[33299]: mon.a calling monitor election 2024-05-03T18:38:44.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:44 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-03T18:38:44.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:44 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-03T18:38:44.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:44 smithi045 ceph-mon[33299]: mon.c calling monitor election 2024-05-03T18:38:44.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:44 smithi045 ceph-mon[33299]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:38:44.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:44 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-03T18:38:44.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:44 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-03T18:38:44.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:44 smithi045 ceph-mon[33299]: mon.b calling monitor election 2024-05-03T18:38:44.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:44 smithi045 ceph-mon[33299]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:38:44.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:44 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-03T18:38:44.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:44 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-03T18:38:44.749 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:44 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-03T18:38:44.749 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:44 smithi045 ceph-mon[33299]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-03T18:38:44.749 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:44 smithi045 ceph-mon[33299]: monmap epoch 3 2024-05-03T18:38:44.749 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:44 smithi045 ceph-mon[33299]: fsid f500455e-097b-11ef-bc97-c7b262605968 2024-05-03T18:38:44.749 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:44 smithi045 ceph-mon[33299]: last_changed 2024-05-03T18:38:39.248717+0000 2024-05-03T18:38:44.749 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:44 smithi045 ceph-mon[33299]: created 2024-05-03T18:36:40.734476+0000 2024-05-03T18:38:44.749 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:44 smithi045 ceph-mon[33299]: min_mon_release 19 (squid) 2024-05-03T18:38:44.749 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:44 smithi045 ceph-mon[33299]: election_strategy: 1 2024-05-03T18:38:44.749 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:44 smithi045 ceph-mon[33299]: 0: [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] mon.a 2024-05-03T18:38:44.749 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:44 smithi045 ceph-mon[33299]: 1: [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] mon.c 2024-05-03T18:38:44.749 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:44 smithi045 ceph-mon[33299]: 2: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2024-05-03T18:38:44.749 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:44 smithi045 ceph-mon[33299]: fsmap 2024-05-03T18:38:44.749 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:44 smithi045 ceph-mon[33299]: osdmap e4: 0 total, 0 up, 0 in 2024-05-03T18:38:44.749 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:44 smithi045 ceph-mon[33299]: mgrmap e13: x(active, since 58s) 2024-05-03T18:38:44.749 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:44 smithi045 ceph-mon[33299]: overall HEALTH_OK 2024-05-03T18:38:44.750 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:44 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:44.750 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:44 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:44.750 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:44 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:44.750 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:44 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:45.282 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-05-03T18:38:45.282 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f500455e-097b-11ef-bc97-c7b262605968 -- ceph config generate-minimal-conf 2024-05-03T18:38:45.538 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:38:45.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:38:45 smithi113 ceph-mon[33540]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:38:45.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:38:45 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:38:45.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:38:45 smithi113 ceph-mon[33540]: from='client.? 172.21.15.113:0/2508921939' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-03T18:38:45.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:38:45 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-03T18:38:45.567 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:45 smithi032 ceph-mon[29763]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:38:45.567 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:45 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:38:45.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:45 smithi032 ceph-mon[29763]: from='client.? 172.21.15.113:0/2508921939' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-03T18:38:45.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:45 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-03T18:38:45.747 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:45 smithi045 ceph-mon[33299]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:38:45.747 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:45 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:38:45.747 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:45 smithi045 ceph-mon[33299]: from='client.? 172.21.15.113:0/2508921939' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-03T18:38:45.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:45 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-03T18:38:46.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:46 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:46.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:46 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:46.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:46 smithi032 ceph-mon[29763]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:38:46.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:46 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:46.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:46 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:46.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:46 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:46.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:46 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:38:46.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:46 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:38:46.819 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:46 smithi032 ceph-mon[29763]: Updating smithi032:/etc/ceph/ceph.conf 2024-05-03T18:38:46.819 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:46 smithi032 ceph-mon[29763]: Updating smithi045:/etc/ceph/ceph.conf 2024-05-03T18:38:46.819 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:46 smithi032 ceph-mon[29763]: Updating smithi113:/etc/ceph/ceph.conf 2024-05-03T18:38:46.819 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:46 smithi032 ceph-mon[29763]: Updating smithi032:/var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-03T18:38:46.819 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:46 smithi032 ceph-mon[29763]: Updating smithi113:/var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-03T18:38:46.819 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:46 smithi032 ceph-mon[29763]: Updating smithi045:/var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-03T18:38:46.819 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:46 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:46.819 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:46 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:46.819 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:46 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:46.819 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:46 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:46.819 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:46 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:46.819 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:46 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:46.819 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:46 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:46.997 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:46 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:46.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:46 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:46.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:46 smithi045 ceph-mon[33299]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:38:46.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:46 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:46.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:46 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:46.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:46 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:46.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:46 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:38:46.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:46 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:38:46.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:46 smithi045 ceph-mon[33299]: Updating smithi032:/etc/ceph/ceph.conf 2024-05-03T18:38:46.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:46 smithi045 ceph-mon[33299]: Updating smithi045:/etc/ceph/ceph.conf 2024-05-03T18:38:46.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:46 smithi045 ceph-mon[33299]: Updating smithi113:/etc/ceph/ceph.conf 2024-05-03T18:38:46.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:46 smithi045 ceph-mon[33299]: Updating smithi032:/var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-03T18:38:46.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:46 smithi045 ceph-mon[33299]: Updating smithi113:/var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-03T18:38:46.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:46 smithi045 ceph-mon[33299]: Updating smithi045:/var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-03T18:38:46.999 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:46 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:46.999 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:46 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:46.999 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:46 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:46.999 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:46 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:46.999 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:46 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:46.999 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:46 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:46.999 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:46 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:47.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:38:46 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:47.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:38:46 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:47.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:38:46 smithi113 ceph-mon[33540]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:38:47.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:38:46 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:47.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:38:46 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:47.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:38:46 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:47.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:38:46 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:38:47.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:38:46 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:38:47.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:38:46 smithi113 ceph-mon[33540]: Updating smithi032:/etc/ceph/ceph.conf 2024-05-03T18:38:47.052 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:38:46 smithi113 ceph-mon[33540]: Updating smithi045:/etc/ceph/ceph.conf 2024-05-03T18:38:47.052 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:38:46 smithi113 ceph-mon[33540]: Updating smithi113:/etc/ceph/ceph.conf 2024-05-03T18:38:47.052 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:38:46 smithi113 ceph-mon[33540]: Updating smithi032:/var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-03T18:38:47.052 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:38:46 smithi113 ceph-mon[33540]: Updating smithi113:/var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-03T18:38:47.052 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:38:46 smithi113 ceph-mon[33540]: Updating smithi045:/var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-03T18:38:47.052 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:38:46 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:47.052 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:38:46 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:47.054 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:38:46 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:47.054 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:38:46 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:47.054 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:38:46 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:47.054 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:38:46 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:47.054 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:38:46 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:47.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:47.277+0000 7f4c38f01700 1 -- 172.21.15.32:0/44832350 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4c34103110 msgr2=0x7f4c341034f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:38:47.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:47.277+0000 7f4c38f01700 1 --2- 172.21.15.32:0/44832350 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4c34103110 0x7f4c341034f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f4c24009910 tx=0x7f4c24009c20 comp rx=0 tx=0).stop 2024-05-03T18:38:47.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:47.277+0000 7f4c38f01700 1 -- 172.21.15.32:0/44832350 shutdown_connections 2024-05-03T18:38:47.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:47.277+0000 7f4c38f01700 1 --2- 172.21.15.32:0/44832350 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4c34103110 0x7f4c341034f0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:47.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:47.277+0000 7f4c38f01700 1 -- 172.21.15.32:0/44832350 >> 172.21.15.32:0/44832350 conn(0x7f4c340fe9c0 msgr2=0x7f4c34100de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:38:47.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:47.277+0000 7f4c38f01700 1 -- 172.21.15.32:0/44832350 shutdown_connections 2024-05-03T18:38:47.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:47.277+0000 7f4c38f01700 1 -- 172.21.15.32:0/44832350 wait complete. 2024-05-03T18:38:47.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:47.278+0000 7f4c38f01700 1 Processor -- start 2024-05-03T18:38:47.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:47.278+0000 7f4c38f01700 1 -- start start 2024-05-03T18:38:47.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:47.278+0000 7f4c38f01700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c34103110 0x7f4c340757a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:38:47.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:47.278+0000 7f4c38f01700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4c340773e0 0x7f4c34075ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:38:47.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:47.278+0000 7f4c38f01700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4c34076220 0x7f4c341ab510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:38:47.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:47.278+0000 7f4c38f01700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f4c3410d390 con 0x7f4c34076220 2024-05-03T18:38:47.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:47.278+0000 7f4c38f01700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4c3410d210 con 0x7f4c34103110 2024-05-03T18:38:47.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:47.278+0000 7f4c38f01700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f4c3410d510 con 0x7f4c340773e0 2024-05-03T18:38:47.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:47.278+0000 7f4c32d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4c34076220 0x7f4c341ab510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:38:47.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:47.278+0000 7f4c32d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4c34076220 0x7f4c341ab510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38590/0 (socket says 172.21.15.32:38590) 2024-05-03T18:38:47.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:47.278+0000 7f4c32d9d700 1 -- 172.21.15.32:0/3721328396 learned_addr learned my addr 172.21.15.32:0/3721328396 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:38:47.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:47.278+0000 7f4c31d9b700 1 --2- 172.21.15.32:0/3721328396 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4c340773e0 0x7f4c34075ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:38:47.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:47.279+0000 7f4c3259c700 1 --2- 172.21.15.32:0/3721328396 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c34103110 0x7f4c340757a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:38:47.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:47.279+0000 7f4c32d9d700 1 -- 172.21.15.32:0/3721328396 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4c340773e0 msgr2=0x7f4c34075ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:38:47.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:47.279+0000 7f4c32d9d700 1 --2- 172.21.15.32:0/3721328396 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4c340773e0 0x7f4c34075ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:47.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:47.279+0000 7f4c32d9d700 1 -- 172.21.15.32:0/3721328396 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c34103110 msgr2=0x7f4c340757a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:38:47.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:47.279+0000 7f4c32d9d700 1 --2- 172.21.15.32:0/3721328396 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c34103110 0x7f4c340757a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:47.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:47.279+0000 7f4c32d9d700 1 -- 172.21.15.32:0/3721328396 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c341abc60 con 0x7f4c34076220 2024-05-03T18:38:47.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:47.279+0000 7f4c3259c700 1 --2- 172.21.15.32:0/3721328396 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c34103110 0x7f4c340757a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T18:38:47.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:47.279+0000 7f4c32d9d700 1 --2- 172.21.15.32:0/3721328396 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4c34076220 0x7f4c341ab510 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c00ba50 tx=0x7f4c2c00bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:38:47.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:47.279+0000 7f4c237fe700 1 -- 172.21.15.32:0/3721328396 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c2c019070 con 0x7f4c34076220 2024-05-03T18:38:47.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:47.279+0000 7f4c237fe700 1 -- 172.21.15.32:0/3721328396 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c2c014440 con 0x7f4c34076220 2024-05-03T18:38:47.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:47.279+0000 7f4c237fe700 1 -- 172.21.15.32:0/3721328396 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c2c00ce60 con 0x7f4c34076220 2024-05-03T18:38:47.282 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:47.279+0000 7f4c38f01700 1 -- 172.21.15.32:0/3721328396 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c341abf50 con 0x7f4c34076220 2024-05-03T18:38:47.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:47.280+0000 7f4c38f01700 1 -- 172.21.15.32:0/3721328396 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c341ac3f0 con 0x7f4c34076220 2024-05-03T18:38:47.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:47.281+0000 7f4c237fe700 1 -- 172.21.15.32:0/3721328396 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f4c2c01c070 con 0x7f4c34076220 2024-05-03T18:38:47.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:47.281+0000 7f4c38f01700 1 -- 172.21.15.32:0/3721328396 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c341045f0 con 0x7f4c34076220 2024-05-03T18:38:47.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:47.281+0000 7f4c237fe700 1 --2- 172.21.15.32:0/3721328396 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f4c1c03e9a0 0x7f4c1c040e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:38:47.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:47.281+0000 7f4c237fe700 1 -- 172.21.15.32:0/3721328396 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f4c2c051d10 con 0x7f4c34076220 2024-05-03T18:38:47.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:47.281+0000 7f4c3259c700 1 --2- 172.21.15.32:0/3721328396 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f4c1c03e9a0 0x7f4c1c040e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:38:47.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:47.282+0000 7f4c3259c700 1 --2- 172.21.15.32:0/3721328396 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f4c1c03e9a0 0x7f4c1c040e60 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f4c24006010 tx=0x7f4c2401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:38:47.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:47.288+0000 7f4c237fe700 1 -- 172.21.15.32:0/3721328396 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f4c2c014710 con 0x7f4c34076220 2024-05-03T18:38:47.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:47.515+0000 7f4c38f01700 1 -- 172.21.15.32:0/3721328396 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f4c341ac770 con 0x7f4c34076220 2024-05-03T18:38:47.517 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:47.515+0000 7f4c237fe700 1 -- 172.21.15.32:0/3721328396 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+273 (secure 0 0 0) 0x7f4c2c022170 con 0x7f4c34076220 2024-05-03T18:38:47.517 INFO:teuthology.orchestra.run.smithi032.stdout:# minimal ceph.conf for f500455e-097b-11ef-bc97-c7b262605968 2024-05-03T18:38:47.517 INFO:teuthology.orchestra.run.smithi032.stdout:[global] 2024-05-03T18:38:47.517 INFO:teuthology.orchestra.run.smithi032.stdout: fsid = f500455e-097b-11ef-bc97-c7b262605968 2024-05-03T18:38:47.517 INFO:teuthology.orchestra.run.smithi032.stdout: mon_host = [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] 2024-05-03T18:38:47.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:47.518+0000 7f4c38f01700 1 -- 172.21.15.32:0/3721328396 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f4c1c03e9a0 msgr2=0x7f4c1c040e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:38:47.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:47.518+0000 7f4c38f01700 1 --2- 172.21.15.32:0/3721328396 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f4c1c03e9a0 0x7f4c1c040e60 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f4c24006010 tx=0x7f4c2401a040 comp rx=0 tx=0).stop 2024-05-03T18:38:47.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:47.518+0000 7f4c38f01700 1 -- 172.21.15.32:0/3721328396 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4c34076220 msgr2=0x7f4c341ab510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:38:47.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:47.518+0000 7f4c38f01700 1 --2- 172.21.15.32:0/3721328396 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4c34076220 0x7f4c341ab510 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c00ba50 tx=0x7f4c2c00bd60 comp rx=0 tx=0).stop 2024-05-03T18:38:47.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:47.518+0000 7f4c38f01700 1 -- 172.21.15.32:0/3721328396 shutdown_connections 2024-05-03T18:38:47.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:47.518+0000 7f4c38f01700 1 --2- 172.21.15.32:0/3721328396 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f4c1c03e9a0 0x7f4c1c040e60 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:47.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:47.518+0000 7f4c38f01700 1 --2- 172.21.15.32:0/3721328396 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4c34076220 0x7f4c341ab510 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:47.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:47.518+0000 7f4c38f01700 1 --2- 172.21.15.32:0/3721328396 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c34103110 0x7f4c340757a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:47.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:47.518+0000 7f4c38f01700 1 --2- 172.21.15.32:0/3721328396 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4c340773e0 0x7f4c34075ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:47.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:47.518+0000 7f4c38f01700 1 -- 172.21.15.32:0/3721328396 >> 172.21.15.32:0/3721328396 conn(0x7f4c340fe9c0 msgr2=0x7f4c34104ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:38:47.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:47.518+0000 7f4c38f01700 1 -- 172.21.15.32:0/3721328396 shutdown_connections 2024-05-03T18:38:47.519 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:47.518+0000 7f4c38f01700 1 -- 172.21.15.32:0/3721328396 wait complete. 2024-05-03T18:38:47.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:47 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:47.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:47 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:47.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:47 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:47.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:47 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:47.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:47 smithi032 ceph-mon[29763]: Reconfiguring mon.a (unknown last config time)... 2024-05-03T18:38:47.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:47 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-03T18:38:47.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:47 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-03T18:38:47.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:47 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:38:47.819 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:47 smithi032 ceph-mon[29763]: Reconfiguring daemon mon.a on smithi032 2024-05-03T18:38:47.819 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:47 smithi032 ceph-mon[29763]: from='client.? 172.21.15.32:0/3721328396' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:38:47.997 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:47 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:47.997 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:47 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:47.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:47 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:47.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:47 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:47.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:47 smithi045 ceph-mon[33299]: Reconfiguring mon.a (unknown last config time)... 2024-05-03T18:38:47.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:47 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-03T18:38:47.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:47 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-03T18:38:47.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:47 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:38:47.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:47 smithi045 ceph-mon[33299]: Reconfiguring daemon mon.a on smithi032 2024-05-03T18:38:47.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:47 smithi045 ceph-mon[33299]: from='client.? 172.21.15.32:0/3721328396' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:38:48.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:38:47 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:48.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:38:47 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:48.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:38:47 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:48.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:38:47 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:48.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:38:47 smithi113 ceph-mon[33540]: Reconfiguring mon.a (unknown last config time)... 2024-05-03T18:38:48.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:38:47 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-03T18:38:48.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:38:47 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-03T18:38:48.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:38:47 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:38:48.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:38:47 smithi113 ceph-mon[33540]: Reconfiguring daemon mon.a on smithi032 2024-05-03T18:38:48.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:38:47 smithi113 ceph-mon[33540]: from='client.? 172.21.15.32:0/3721328396' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:38:48.213 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-05-03T18:38:48.213 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-05-03T18:38:48.213 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd of=/etc/ceph/ceph.conf 2024-05-03T18:38:48.240 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-05-03T18:38:48.240 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-03T18:38:48.305 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-05-03T18:38:48.305 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/etc/ceph/ceph.conf 2024-05-03T18:38:48.338 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-05-03T18:38:48.338 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-03T18:38:48.407 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-05-03T18:38:48.408 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd of=/etc/ceph/ceph.conf 2024-05-03T18:38:48.438 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-05-03T18:38:48.438 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-03T18:38:48.506 INFO:tasks.cephadm:Adding mgr.x on smithi032 2024-05-03T18:38:48.506 INFO:tasks.cephadm:Adding mgr.y on smithi045 2024-05-03T18:38:48.507 INFO:tasks.cephadm:Adding mgr.z on smithi113 2024-05-03T18:38:48.507 DEBUG:teuthology.orchestra.run.smithi113:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f500455e-097b-11ef-bc97-c7b262605968 -- ceph orch apply mgr '3;smithi032=x;smithi045=y;smithi113=z' 2024-05-03T18:38:48.783 INFO:teuthology.orchestra.run.smithi113.stderr:Inferring config /var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/mon.c/config 2024-05-03T18:38:48.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:48 smithi032 ceph-mon[29763]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:38:48.997 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:48 smithi045 ceph-mon[33299]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:38:49.050 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:38:48 smithi113 ceph-mon[33540]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:38:50.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:50.127+0000 7f8b9f60c700 1 -- 172.21.15.113:0/963335000 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8b98103010 msgr2=0x7f8b98103430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:38:50.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:50.127+0000 7f8b9f60c700 1 --2- 172.21.15.113:0/963335000 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8b98103010 0x7f8b98103430 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f8b94009910 tx=0x7f8b94009c20 comp rx=0 tx=0).stop 2024-05-03T18:38:50.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:50.127+0000 7f8b9f60c700 1 -- 172.21.15.113:0/963335000 shutdown_connections 2024-05-03T18:38:50.131 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:50.127+0000 7f8b9f60c700 1 --2- 172.21.15.113:0/963335000 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8b98103010 0x7f8b98103430 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:50.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:50.127+0000 7f8b9f60c700 1 -- 172.21.15.113:0/963335000 >> 172.21.15.113:0/963335000 conn(0x7f8b980fe5d0 msgr2=0x7f8b981009f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:38:50.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:50.127+0000 7f8b9f60c700 1 -- 172.21.15.113:0/963335000 shutdown_connections 2024-05-03T18:38:50.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:50.127+0000 7f8b9f60c700 1 -- 172.21.15.113:0/963335000 wait complete. 2024-05-03T18:38:50.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:50.127+0000 7f8b9f60c700 1 Processor -- start 2024-05-03T18:38:50.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:50.127+0000 7f8b9f60c700 1 -- start start 2024-05-03T18:38:50.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:50.128+0000 7f8b9f60c700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8b98103010 0x7f8b98198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:38:50.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:50.128+0000 7f8b9f60c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8b98198ca0 0x7f8b9819dd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:38:50.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:50.128+0000 7f8b9f60c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b98199120 0x7f8b981a02a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:38:50.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:50.128+0000 7f8b9f60c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f8b98108f50 con 0x7f8b98198ca0 2024-05-03T18:38:50.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:50.128+0000 7f8b9f60c700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8b98108dd0 con 0x7f8b98199120 2024-05-03T18:38:50.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:50.128+0000 7f8b9f60c700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f8b981090d0 con 0x7f8b98103010 2024-05-03T18:38:50.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:50.128+0000 7f8b9d3a8700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8b98103010 0x7f8b98198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:38:50.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:50.128+0000 7f8b9d3a8700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8b98103010 0x7f8b98198760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:46718/0 (socket says 172.21.15.113:46718) 2024-05-03T18:38:50.132 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:50.128+0000 7f8b9d3a8700 1 -- 172.21.15.113:0/646708289 learned_addr learned my addr 172.21.15.113:0/646708289 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-05-03T18:38:50.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:50.128+0000 7f8b9cba7700 1 --2- 172.21.15.113:0/646708289 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8b98198ca0 0x7f8b9819dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:38:50.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:50.128+0000 7f8b9dba9700 1 --2- 172.21.15.113:0/646708289 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b98199120 0x7f8b981a02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:38:50.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:50.128+0000 7f8b9d3a8700 1 -- 172.21.15.113:0/646708289 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b98199120 msgr2=0x7f8b981a02a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:38:50.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:50.128+0000 7f8b9d3a8700 1 --2- 172.21.15.113:0/646708289 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b98199120 0x7f8b981a02a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:50.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:50.128+0000 7f8b9d3a8700 1 -- 172.21.15.113:0/646708289 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8b98198ca0 msgr2=0x7f8b9819dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:38:50.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:50.128+0000 7f8b9d3a8700 1 --2- 172.21.15.113:0/646708289 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8b98198ca0 0x7f8b9819dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:50.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:50.128+0000 7f8b9d3a8700 1 -- 172.21.15.113:0/646708289 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b981a0b70 con 0x7f8b98103010 2024-05-03T18:38:50.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:50.128+0000 7f8b9d3a8700 1 --2- 172.21.15.113:0/646708289 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8b98103010 0x7f8b98198760 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f8b94005e00 tx=0x7f8b94005020 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:38:50.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:50.128+0000 7f8b8e7fc700 1 -- 172.21.15.113:0/646708289 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b94027070 con 0x7f8b98103010 2024-05-03T18:38:50.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:50.128+0000 7f8b8e7fc700 1 -- 172.21.15.113:0/646708289 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8b9400be10 con 0x7f8b98103010 2024-05-03T18:38:50.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:50.128+0000 7f8b9f60c700 1 -- 172.21.15.113:0/646708289 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b981a0e00 con 0x7f8b98103010 2024-05-03T18:38:50.133 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:50.128+0000 7f8b8e7fc700 1 -- 172.21.15.113:0/646708289 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b94022da0 con 0x7f8b98103010 2024-05-03T18:38:50.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:50.128+0000 7f8b9f60c700 1 -- 172.21.15.113:0/646708289 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b981a1270 con 0x7f8b98103010 2024-05-03T18:38:50.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:50.129+0000 7f8b8e7fc700 1 -- 172.21.15.113:0/646708289 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f8b9402a070 con 0x7f8b98103010 2024-05-03T18:38:50.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:50.129+0000 7f8b8e7fc700 1 --2- 172.21.15.113:0/646708289 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f8b8403e9a0 0x7f8b84040e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:38:50.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:50.129+0000 7f8b9f60c700 1 -- 172.21.15.113:0/646708289 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b98064b00 con 0x7f8b98103010 2024-05-03T18:38:50.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:50.129+0000 7f8b8e7fc700 1 -- 172.21.15.113:0/646708289 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f8b94053a20 con 0x7f8b98103010 2024-05-03T18:38:50.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:50.130+0000 7f8b9cba7700 1 --2- 172.21.15.113:0/646708289 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f8b8403e9a0 0x7f8b84040e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:38:50.134 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:50.130+0000 7f8b9cba7700 1 --2- 172.21.15.113:0/646708289 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f8b8403e9a0 0x7f8b84040e60 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f8b88009ad0 tx=0x7f8b88009210 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:38:50.136 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:50.135+0000 7f8b8e7fc700 1 -- 172.21.15.113:0/646708289 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f8b94031070 con 0x7f8b98103010 2024-05-03T18:38:50.247 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:49 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:50.247 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:49 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:50.247 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:49 smithi045 ceph-mon[33299]: Reconfiguring mon.b (monmap changed)... 2024-05-03T18:38:50.248 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:49 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-03T18:38:50.248 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:49 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-03T18:38:50.248 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:49 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:38:50.248 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:49 smithi045 ceph-mon[33299]: Reconfiguring daemon mon.b on smithi045 2024-05-03T18:38:50.300 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:38:49 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:50.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:38:49 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:50.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:38:49 smithi113 ceph-mon[33540]: Reconfiguring mon.b (monmap changed)... 2024-05-03T18:38:50.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:38:49 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-03T18:38:50.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:38:49 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-03T18:38:50.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:38:49 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:38:50.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:38:49 smithi113 ceph-mon[33540]: Reconfiguring daemon mon.b on smithi045 2024-05-03T18:38:50.317 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:49 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:50.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:49 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:50.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:49 smithi032 ceph-mon[29763]: Reconfiguring mon.b (monmap changed)... 2024-05-03T18:38:50.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:49 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-03T18:38:50.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:49 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-03T18:38:50.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:49 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:38:50.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:49 smithi032 ceph-mon[29763]: Reconfiguring daemon mon.b on smithi045 2024-05-03T18:38:50.349 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:50.348+0000 7f8b9f60c700 1 -- 172.21.15.113:0/646708289 --> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi032=x;smithi045=y;smithi113=z", "target": ["mon-mgr", ""]}) -- 0x7f8b98063970 con 0x7f8b8403e9a0 2024-05-03T18:38:50.356 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:50.354+0000 7f8b8e7fc700 1 -- 172.21.15.113:0/646708289 <== mgr.14152 v2:172.21.15.32:6800/1796547904 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f8b98063970 con 0x7f8b8403e9a0 2024-05-03T18:38:50.356 INFO:teuthology.orchestra.run.smithi113.stdout:Scheduled mgr update... 2024-05-03T18:38:50.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:50.356+0000 7f8b9f60c700 1 -- 172.21.15.113:0/646708289 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f8b8403e9a0 msgr2=0x7f8b84040e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:38:50.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:50.356+0000 7f8b9f60c700 1 --2- 172.21.15.113:0/646708289 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f8b8403e9a0 0x7f8b84040e60 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f8b88009ad0 tx=0x7f8b88009210 comp rx=0 tx=0).stop 2024-05-03T18:38:50.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:50.356+0000 7f8b9f60c700 1 -- 172.21.15.113:0/646708289 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8b98103010 msgr2=0x7f8b98198760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:38:50.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:50.356+0000 7f8b9f60c700 1 --2- 172.21.15.113:0/646708289 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8b98103010 0x7f8b98198760 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f8b94005e00 tx=0x7f8b94005020 comp rx=0 tx=0).stop 2024-05-03T18:38:50.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:50.356+0000 7f8b9f60c700 1 -- 172.21.15.113:0/646708289 shutdown_connections 2024-05-03T18:38:50.358 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:50.356+0000 7f8b9f60c700 1 --2- 172.21.15.113:0/646708289 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f8b8403e9a0 0x7f8b84040e60 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:50.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:50.356+0000 7f8b9f60c700 1 --2- 172.21.15.113:0/646708289 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8b98199120 0x7f8b981a02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:50.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:50.356+0000 7f8b9f60c700 1 --2- 172.21.15.113:0/646708289 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8b98103010 0x7f8b98198760 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:50.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:50.356+0000 7f8b9f60c700 1 --2- 172.21.15.113:0/646708289 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8b98198ca0 0x7f8b9819dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:50.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:50.356+0000 7f8b9f60c700 1 -- 172.21.15.113:0/646708289 >> 172.21.15.113:0/646708289 conn(0x7f8b980fe5d0 msgr2=0x7f8b980ff180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:38:50.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:50.356+0000 7f8b9f60c700 1 -- 172.21.15.113:0/646708289 shutdown_connections 2024-05-03T18:38:50.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:38:50.356+0000 7f8b9f60c700 1 -- 172.21.15.113:0/646708289 wait complete. 2024-05-03T18:38:51.009 DEBUG:teuthology.orchestra.run.smithi045:mgr.y> sudo journalctl -f -n 0 -u ceph-f500455e-097b-11ef-bc97-c7b262605968@mgr.y.service 2024-05-03T18:38:51.011 DEBUG:teuthology.orchestra.run.smithi113:mgr.z> sudo journalctl -f -n 0 -u ceph-f500455e-097b-11ef-bc97-c7b262605968@mgr.z.service 2024-05-03T18:38:51.014 INFO:tasks.cephadm:Deploying OSDs... 2024-05-03T18:38:51.014 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-05-03T18:38:51.014 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/scratch_devs of=/dev/stdout 2024-05-03T18:38:51.034 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-03T18:38:51.034 DEBUG:teuthology.orchestra.run.smithi032:> stat /dev/vg_nvme/lv_1 2024-05-03T18:38:51.093 INFO:teuthology.orchestra.run.smithi032.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-03T18:38:51.094 INFO:teuthology.orchestra.run.smithi032.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-03T18:38:51.094 INFO:teuthology.orchestra.run.smithi032.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-05-03T18:38:51.094 INFO:teuthology.orchestra.run.smithi032.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-03T18:38:51.094 INFO:teuthology.orchestra.run.smithi032.stdout:Context: system_u:object_r:device_t:s0 2024-05-03T18:38:51.094 INFO:teuthology.orchestra.run.smithi032.stdout:Access: 2024-05-03 18:37:55.962218854 +0000 2024-05-03T18:38:51.094 INFO:teuthology.orchestra.run.smithi032.stdout:Modify: 2024-05-03 18:33:06.609058371 +0000 2024-05-03T18:38:51.094 INFO:teuthology.orchestra.run.smithi032.stdout:Change: 2024-05-03 18:33:06.609058371 +0000 2024-05-03T18:38:51.094 INFO:teuthology.orchestra.run.smithi032.stdout: Birth: 2024-05-03 18:33:06.609058371 +0000 2024-05-03T18:38:51.094 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-03T18:38:51.162 INFO:teuthology.orchestra.run.smithi032.stderr:1+0 records in 2024-05-03T18:38:51.163 INFO:teuthology.orchestra.run.smithi032.stderr:1+0 records out 2024-05-03T18:38:51.163 INFO:teuthology.orchestra.run.smithi032.stderr:512 bytes copied, 0.00013442 s, 3.8 MB/s 2024-05-03T18:38:51.164 DEBUG:teuthology.orchestra.run.smithi032:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-03T18:38:51.224 DEBUG:teuthology.orchestra.run.smithi032:> stat /dev/vg_nvme/lv_2 2024-05-03T18:38:51.284 INFO:teuthology.orchestra.run.smithi032.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-03T18:38:51.284 INFO:teuthology.orchestra.run.smithi032.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-03T18:38:51.284 INFO:teuthology.orchestra.run.smithi032.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-05-03T18:38:51.284 INFO:teuthology.orchestra.run.smithi032.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-03T18:38:51.284 INFO:teuthology.orchestra.run.smithi032.stdout:Context: system_u:object_r:device_t:s0 2024-05-03T18:38:51.284 INFO:teuthology.orchestra.run.smithi032.stdout:Access: 2024-05-03 18:37:55.998218160 +0000 2024-05-03T18:38:51.284 INFO:teuthology.orchestra.run.smithi032.stdout:Modify: 2024-05-03 18:33:06.945051543 +0000 2024-05-03T18:38:51.284 INFO:teuthology.orchestra.run.smithi032.stdout:Change: 2024-05-03 18:33:06.945051543 +0000 2024-05-03T18:38:51.284 INFO:teuthology.orchestra.run.smithi032.stdout: Birth: 2024-05-03 18:33:06.945051543 +0000 2024-05-03T18:38:51.285 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-03T18:38:51.350 INFO:teuthology.orchestra.run.smithi032.stderr:1+0 records in 2024-05-03T18:38:51.350 INFO:teuthology.orchestra.run.smithi032.stderr:1+0 records out 2024-05-03T18:38:51.350 INFO:teuthology.orchestra.run.smithi032.stderr:512 bytes copied, 0.000114401 s, 4.5 MB/s 2024-05-03T18:38:51.351 DEBUG:teuthology.orchestra.run.smithi032:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-03T18:38:51.409 DEBUG:teuthology.orchestra.run.smithi032:> stat /dev/vg_nvme/lv_3 2024-05-03T18:38:51.466 INFO:teuthology.orchestra.run.smithi032.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-03T18:38:51.466 INFO:teuthology.orchestra.run.smithi032.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-03T18:38:51.466 INFO:teuthology.orchestra.run.smithi032.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-05-03T18:38:51.466 INFO:teuthology.orchestra.run.smithi032.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-03T18:38:51.466 INFO:teuthology.orchestra.run.smithi032.stdout:Context: system_u:object_r:device_t:s0 2024-05-03T18:38:51.466 INFO:teuthology.orchestra.run.smithi032.stdout:Access: 2024-05-03 18:37:56.032217504 +0000 2024-05-03T18:38:51.466 INFO:teuthology.orchestra.run.smithi032.stdout:Modify: 2024-05-03 18:33:07.284044654 +0000 2024-05-03T18:38:51.466 INFO:teuthology.orchestra.run.smithi032.stdout:Change: 2024-05-03 18:33:07.284044654 +0000 2024-05-03T18:38:51.467 INFO:teuthology.orchestra.run.smithi032.stdout: Birth: 2024-05-03 18:33:07.284044654 +0000 2024-05-03T18:38:51.467 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-03T18:38:51.529 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:51 smithi032 ceph-mon[29763]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:38:51.529 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:51 smithi032 ceph-mon[29763]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi032=x;smithi045=y;smithi113=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:38:51.529 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:51 smithi032 ceph-mon[29763]: Saving service mgr spec with placement smithi032=x;smithi045=y;smithi113=z;count:3 2024-05-03T18:38:51.529 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:51 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:51.530 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:51 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:51.530 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:51 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:51.530 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:51 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-03T18:38:51.530 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:51 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-03T18:38:51.530 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:51 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:38:51.532 INFO:teuthology.orchestra.run.smithi032.stderr:1+0 records in 2024-05-03T18:38:51.532 INFO:teuthology.orchestra.run.smithi032.stderr:1+0 records out 2024-05-03T18:38:51.532 INFO:teuthology.orchestra.run.smithi032.stderr:512 bytes copied, 0.000108881 s, 4.7 MB/s 2024-05-03T18:38:51.533 DEBUG:teuthology.orchestra.run.smithi032:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-03T18:38:51.591 DEBUG:teuthology.orchestra.run.smithi032:> stat /dev/vg_nvme/lv_4 2024-05-03T18:38:51.642 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:38:51 smithi113 ceph-mon[33540]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:38:51.642 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:38:51 smithi113 ceph-mon[33540]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi032=x;smithi045=y;smithi113=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:38:51.642 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:38:51 smithi113 ceph-mon[33540]: Saving service mgr spec with placement smithi032=x;smithi045=y;smithi113=z;count:3 2024-05-03T18:38:51.642 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:38:51 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:51.642 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:38:51 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:51.642 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:38:51 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:51.642 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:38:51 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-03T18:38:51.642 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:38:51 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-03T18:38:51.643 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:38:51 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:38:51.647 INFO:teuthology.orchestra.run.smithi032.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-03T18:38:51.647 INFO:teuthology.orchestra.run.smithi032.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-03T18:38:51.648 INFO:teuthology.orchestra.run.smithi032.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-05-03T18:38:51.648 INFO:teuthology.orchestra.run.smithi032.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-03T18:38:51.648 INFO:teuthology.orchestra.run.smithi032.stdout:Context: system_u:object_r:device_t:s0 2024-05-03T18:38:51.648 INFO:teuthology.orchestra.run.smithi032.stdout:Access: 2024-05-03 18:37:56.072216732 +0000 2024-05-03T18:38:51.648 INFO:teuthology.orchestra.run.smithi032.stdout:Modify: 2024-05-03 18:33:07.633037561 +0000 2024-05-03T18:38:51.648 INFO:teuthology.orchestra.run.smithi032.stdout:Change: 2024-05-03 18:33:07.633037561 +0000 2024-05-03T18:38:51.648 INFO:teuthology.orchestra.run.smithi032.stdout: Birth: 2024-05-03 18:33:07.633037561 +0000 2024-05-03T18:38:51.648 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-03T18:38:51.712 INFO:teuthology.orchestra.run.smithi032.stderr:1+0 records in 2024-05-03T18:38:51.712 INFO:teuthology.orchestra.run.smithi032.stderr:1+0 records out 2024-05-03T18:38:51.712 INFO:teuthology.orchestra.run.smithi032.stderr:512 bytes copied, 0.000115023 s, 4.5 MB/s 2024-05-03T18:38:51.713 DEBUG:teuthology.orchestra.run.smithi032:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-03T18:38:51.747 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:51 smithi045 ceph-mon[33299]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:38:51.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:51 smithi045 ceph-mon[33299]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi032=x;smithi045=y;smithi113=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:38:51.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:51 smithi045 ceph-mon[33299]: Saving service mgr spec with placement smithi032=x;smithi045=y;smithi113=z;count:3 2024-05-03T18:38:51.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:51 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:51.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:51 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:51.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:51 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:51.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:51 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-03T18:38:51.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:51 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-03T18:38:51.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:51 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:38:51.770 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-05-03T18:38:51.770 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/scratch_devs of=/dev/stdout 2024-05-03T18:38:51.788 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-03T18:38:51.789 DEBUG:teuthology.orchestra.run.smithi045:> stat /dev/vg_nvme/lv_1 2024-05-03T18:38:51.845 INFO:teuthology.orchestra.run.smithi045.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-03T18:38:51.845 INFO:teuthology.orchestra.run.smithi045.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-03T18:38:51.846 INFO:teuthology.orchestra.run.smithi045.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-05-03T18:38:51.846 INFO:teuthology.orchestra.run.smithi045.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-03T18:38:51.846 INFO:teuthology.orchestra.run.smithi045.stdout:Context: system_u:object_r:device_t:s0 2024-05-03T18:38:51.846 INFO:teuthology.orchestra.run.smithi045.stdout:Access: 2024-05-03 18:38:38.697871665 +0000 2024-05-03T18:38:51.846 INFO:teuthology.orchestra.run.smithi045.stdout:Modify: 2024-05-03 18:33:41.558399159 +0000 2024-05-03T18:38:51.846 INFO:teuthology.orchestra.run.smithi045.stdout:Change: 2024-05-03 18:33:41.558399159 +0000 2024-05-03T18:38:51.846 INFO:teuthology.orchestra.run.smithi045.stdout: Birth: 2024-05-03 18:33:41.558399159 +0000 2024-05-03T18:38:51.846 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-03T18:38:51.911 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-05-03T18:38:51.911 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-05-03T18:38:51.911 INFO:teuthology.orchestra.run.smithi045.stderr:512 bytes copied, 0.000122754 s, 4.2 MB/s 2024-05-03T18:38:51.912 DEBUG:teuthology.orchestra.run.smithi045:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-03T18:38:51.970 DEBUG:teuthology.orchestra.run.smithi045:> stat /dev/vg_nvme/lv_2 2024-05-03T18:38:52.027 INFO:teuthology.orchestra.run.smithi045.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-03T18:38:52.027 INFO:teuthology.orchestra.run.smithi045.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-03T18:38:52.027 INFO:teuthology.orchestra.run.smithi045.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-05-03T18:38:52.027 INFO:teuthology.orchestra.run.smithi045.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-03T18:38:52.027 INFO:teuthology.orchestra.run.smithi045.stdout:Context: system_u:object_r:device_t:s0 2024-05-03T18:38:52.028 INFO:teuthology.orchestra.run.smithi045.stdout:Access: 2024-05-03 18:38:38.698871650 +0000 2024-05-03T18:38:52.028 INFO:teuthology.orchestra.run.smithi045.stdout:Modify: 2024-05-03 18:33:41.890393802 +0000 2024-05-03T18:38:52.028 INFO:teuthology.orchestra.run.smithi045.stdout:Change: 2024-05-03 18:33:41.890393802 +0000 2024-05-03T18:38:52.028 INFO:teuthology.orchestra.run.smithi045.stdout: Birth: 2024-05-03 18:33:41.890393802 +0000 2024-05-03T18:38:52.028 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-03T18:38:52.092 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-05-03T18:38:52.092 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-05-03T18:38:52.092 INFO:teuthology.orchestra.run.smithi045.stderr:512 bytes copied, 0.000116409 s, 4.4 MB/s 2024-05-03T18:38:52.093 DEBUG:teuthology.orchestra.run.smithi045:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-03T18:38:52.151 DEBUG:teuthology.orchestra.run.smithi045:> stat /dev/vg_nvme/lv_3 2024-05-03T18:38:52.208 INFO:teuthology.orchestra.run.smithi045.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-03T18:38:52.208 INFO:teuthology.orchestra.run.smithi045.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-03T18:38:52.208 INFO:teuthology.orchestra.run.smithi045.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-05-03T18:38:52.208 INFO:teuthology.orchestra.run.smithi045.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-03T18:38:52.208 INFO:teuthology.orchestra.run.smithi045.stdout:Context: system_u:object_r:device_t:s0 2024-05-03T18:38:52.208 INFO:teuthology.orchestra.run.smithi045.stdout:Access: 2024-05-03 18:38:38.698871650 +0000 2024-05-03T18:38:52.209 INFO:teuthology.orchestra.run.smithi045.stdout:Modify: 2024-05-03 18:33:42.230388316 +0000 2024-05-03T18:38:52.209 INFO:teuthology.orchestra.run.smithi045.stdout:Change: 2024-05-03 18:33:42.230388316 +0000 2024-05-03T18:38:52.209 INFO:teuthology.orchestra.run.smithi045.stdout: Birth: 2024-05-03 18:33:42.230388316 +0000 2024-05-03T18:38:52.209 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-03T18:38:52.273 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-05-03T18:38:52.274 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-05-03T18:38:52.274 INFO:teuthology.orchestra.run.smithi045.stderr:512 bytes copied, 0.000117887 s, 4.3 MB/s 2024-05-03T18:38:52.275 DEBUG:teuthology.orchestra.run.smithi045:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-03T18:38:52.333 DEBUG:teuthology.orchestra.run.smithi045:> stat /dev/vg_nvme/lv_4 2024-05-03T18:38:52.390 INFO:teuthology.orchestra.run.smithi045.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-03T18:38:52.390 INFO:teuthology.orchestra.run.smithi045.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-03T18:38:52.390 INFO:teuthology.orchestra.run.smithi045.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-05-03T18:38:52.390 INFO:teuthology.orchestra.run.smithi045.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-03T18:38:52.390 INFO:teuthology.orchestra.run.smithi045.stdout:Context: system_u:object_r:device_t:s0 2024-05-03T18:38:52.390 INFO:teuthology.orchestra.run.smithi045.stdout:Access: 2024-05-03 18:38:38.699871634 +0000 2024-05-03T18:38:52.391 INFO:teuthology.orchestra.run.smithi045.stdout:Modify: 2024-05-03 18:33:42.567382878 +0000 2024-05-03T18:38:52.391 INFO:teuthology.orchestra.run.smithi045.stdout:Change: 2024-05-03 18:33:42.567382878 +0000 2024-05-03T18:38:52.391 INFO:teuthology.orchestra.run.smithi045.stdout: Birth: 2024-05-03 18:33:42.567382878 +0000 2024-05-03T18:38:52.391 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-03T18:38:52.455 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-05-03T18:38:52.455 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-05-03T18:38:52.455 INFO:teuthology.orchestra.run.smithi045.stderr:512 bytes copied, 0.000113618 s, 4.5 MB/s 2024-05-03T18:38:52.456 DEBUG:teuthology.orchestra.run.smithi045:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-03T18:38:52.513 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-05-03T18:38:52.514 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/scratch_devs of=/dev/stdout 2024-05-03T18:38:52.532 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-03T18:38:52.532 DEBUG:teuthology.orchestra.run.smithi113:> stat /dev/vg_nvme/lv_1 2024-05-03T18:38:52.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:38:52 smithi113 ceph-mon[33540]: Reconfiguring mon.c (monmap changed)... 2024-05-03T18:38:52.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:38:52 smithi113 ceph-mon[33540]: Reconfiguring daemon mon.c on smithi113 2024-05-03T18:38:52.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:38:52 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:52.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:38:52 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:52.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:38:52 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:38:52.567 INFO:teuthology.orchestra.run.smithi113.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-03T18:38:52.567 INFO:teuthology.orchestra.run.smithi113.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-03T18:38:52.567 INFO:teuthology.orchestra.run.smithi113.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-05-03T18:38:52.567 INFO:teuthology.orchestra.run.smithi113.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-03T18:38:52.567 INFO:teuthology.orchestra.run.smithi113.stdout:Context: system_u:object_r:device_t:s0 2024-05-03T18:38:52.567 INFO:teuthology.orchestra.run.smithi113.stdout:Access: 2024-05-03 18:38:29.093292089 +0000 2024-05-03T18:38:52.567 INFO:teuthology.orchestra.run.smithi113.stdout:Modify: 2024-05-03 18:33:16.459777805 +0000 2024-05-03T18:38:52.567 INFO:teuthology.orchestra.run.smithi113.stdout:Change: 2024-05-03 18:33:16.459777805 +0000 2024-05-03T18:38:52.568 INFO:teuthology.orchestra.run.smithi113.stdout: Birth: 2024-05-03 18:33:16.459777805 +0000 2024-05-03T18:38:52.568 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-03T18:38:52.633 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-05-03T18:38:52.633 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-05-03T18:38:52.633 INFO:teuthology.orchestra.run.smithi113.stderr:512 bytes copied, 0.000228884 s, 2.2 MB/s 2024-05-03T18:38:52.635 DEBUG:teuthology.orchestra.run.smithi113:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-03T18:38:52.693 DEBUG:teuthology.orchestra.run.smithi113:> stat /dev/vg_nvme/lv_2 2024-05-03T18:38:52.747 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:52 smithi045 ceph-mon[33299]: Reconfiguring mon.c (monmap changed)... 2024-05-03T18:38:52.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:52 smithi045 ceph-mon[33299]: Reconfiguring daemon mon.c on smithi113 2024-05-03T18:38:52.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:52 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:52.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:52 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:52.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:52 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:38:52.749 INFO:teuthology.orchestra.run.smithi113.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-03T18:38:52.749 INFO:teuthology.orchestra.run.smithi113.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-03T18:38:52.749 INFO:teuthology.orchestra.run.smithi113.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-05-03T18:38:52.750 INFO:teuthology.orchestra.run.smithi113.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-03T18:38:52.750 INFO:teuthology.orchestra.run.smithi113.stdout:Context: system_u:object_r:device_t:s0 2024-05-03T18:38:52.750 INFO:teuthology.orchestra.run.smithi113.stdout:Access: 2024-05-03 18:38:29.093292089 +0000 2024-05-03T18:38:52.750 INFO:teuthology.orchestra.run.smithi113.stdout:Modify: 2024-05-03 18:33:16.799770277 +0000 2024-05-03T18:38:52.750 INFO:teuthology.orchestra.run.smithi113.stdout:Change: 2024-05-03 18:33:16.799770277 +0000 2024-05-03T18:38:52.750 INFO:teuthology.orchestra.run.smithi113.stdout: Birth: 2024-05-03 18:33:16.799770277 +0000 2024-05-03T18:38:52.750 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-03T18:38:52.813 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-05-03T18:38:52.813 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-05-03T18:38:52.813 INFO:teuthology.orchestra.run.smithi113.stderr:512 bytes copied, 0.000214896 s, 2.4 MB/s 2024-05-03T18:38:52.815 DEBUG:teuthology.orchestra.run.smithi113:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-03T18:38:52.817 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:52 smithi032 ceph-mon[29763]: Reconfiguring mon.c (monmap changed)... 2024-05-03T18:38:52.817 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:52 smithi032 ceph-mon[29763]: Reconfiguring daemon mon.c on smithi113 2024-05-03T18:38:52.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:52 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:52.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:52 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:52.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:52 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:38:52.871 DEBUG:teuthology.orchestra.run.smithi113:> stat /dev/vg_nvme/lv_3 2024-05-03T18:38:52.927 INFO:teuthology.orchestra.run.smithi113.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-03T18:38:52.928 INFO:teuthology.orchestra.run.smithi113.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-03T18:38:52.928 INFO:teuthology.orchestra.run.smithi113.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-05-03T18:38:52.928 INFO:teuthology.orchestra.run.smithi113.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-03T18:38:52.928 INFO:teuthology.orchestra.run.smithi113.stdout:Context: system_u:object_r:device_t:s0 2024-05-03T18:38:52.928 INFO:teuthology.orchestra.run.smithi113.stdout:Access: 2024-05-03 18:38:29.093292089 +0000 2024-05-03T18:38:52.928 INFO:teuthology.orchestra.run.smithi113.stdout:Modify: 2024-05-03 18:33:17.146762594 +0000 2024-05-03T18:38:52.928 INFO:teuthology.orchestra.run.smithi113.stdout:Change: 2024-05-03 18:33:17.146762594 +0000 2024-05-03T18:38:52.928 INFO:teuthology.orchestra.run.smithi113.stdout: Birth: 2024-05-03 18:33:17.146762594 +0000 2024-05-03T18:38:52.928 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-03T18:38:52.990 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-05-03T18:38:52.990 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-05-03T18:38:52.990 INFO:teuthology.orchestra.run.smithi113.stderr:512 bytes copied, 0.000191116 s, 2.7 MB/s 2024-05-03T18:38:52.992 DEBUG:teuthology.orchestra.run.smithi113:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-03T18:38:53.048 DEBUG:teuthology.orchestra.run.smithi113:> stat /dev/vg_nvme/lv_4 2024-05-03T18:38:53.103 INFO:teuthology.orchestra.run.smithi113.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-03T18:38:53.103 INFO:teuthology.orchestra.run.smithi113.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-03T18:38:53.103 INFO:teuthology.orchestra.run.smithi113.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-05-03T18:38:53.103 INFO:teuthology.orchestra.run.smithi113.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-03T18:38:53.104 INFO:teuthology.orchestra.run.smithi113.stdout:Context: system_u:object_r:device_t:s0 2024-05-03T18:38:53.104 INFO:teuthology.orchestra.run.smithi113.stdout:Access: 2024-05-03 18:38:29.094292068 +0000 2024-05-03T18:38:53.104 INFO:teuthology.orchestra.run.smithi113.stdout:Modify: 2024-05-03 18:33:17.482755155 +0000 2024-05-03T18:38:53.104 INFO:teuthology.orchestra.run.smithi113.stdout:Change: 2024-05-03 18:33:17.482755155 +0000 2024-05-03T18:38:53.104 INFO:teuthology.orchestra.run.smithi113.stdout: Birth: 2024-05-03 18:33:17.482755155 +0000 2024-05-03T18:38:53.104 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-03T18:38:53.165 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-05-03T18:38:53.165 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-05-03T18:38:53.165 INFO:teuthology.orchestra.run.smithi113.stderr:512 bytes copied, 0.000213648 s, 2.4 MB/s 2024-05-03T18:38:53.167 DEBUG:teuthology.orchestra.run.smithi113:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-03T18:38:53.223 INFO:tasks.cephadm:Deploying osd.0 on smithi032 with /dev/vg_nvme/lv_4... 2024-05-03T18:38:53.223 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f500455e-097b-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-03T18:38:53.465 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:38:53.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:53 smithi032 ceph-mon[29763]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:38:53.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:53 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:38:53.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:53 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:38:53.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:53 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:53.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:53 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-03T18:38:53.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:53 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-03T18:38:53.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:53 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-03T18:38:53.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:53 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:38:53.654 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:38:53 smithi113 ceph-mon[33540]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:38:53.654 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:38:53 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:38:53.655 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:38:53 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:38:53.655 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:38:53 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:53.655 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:38:53 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-03T18:38:53.655 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:38:53 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-03T18:38:53.655 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:38:53 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-03T18:38:53.655 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:38:53 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:38:53.747 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:53 smithi045 ceph-mon[33299]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:38:53.747 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:53 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:38:53.747 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:53 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:38:53.747 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:53 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:53.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:53 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-03T18:38:53.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:53 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-03T18:38:53.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:53 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-03T18:38:53.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:53 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:38:54.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:54 smithi032 ceph-mon[29763]: Deploying daemon mgr.z on smithi113 2024-05-03T18:38:54.606 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:38:54 smithi113 ceph-mon[33540]: Deploying daemon mgr.z on smithi113 2024-05-03T18:38:54.747 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:54 smithi045 ceph-mon[33299]: Deploying daemon mgr.z on smithi113 2024-05-03T18:38:55.002 INFO:journalctl@ceph.mgr.z.smithi113.stdout:May 03 18:38:54 smithi113 systemd[1]: Starting Ceph mgr.z for f500455e-097b-11ef-bc97-c7b262605968... 2024-05-03T18:38:55.253 INFO:journalctl@ceph.mgr.z.smithi113.stdout:May 03 18:38:55 smithi113 podman[35394]: 2024-05-03 18:38:54.90513399 +0000 UTC m=+0.017897208 image pull ee4d846202496ba6096593e8e8becc0700a5eb7ebd95536031187f5f337d4012 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd 2024-05-03T18:38:55.253 INFO:journalctl@ceph.mgr.z.smithi113.stdout:May 03 18:38:55 smithi113 podman[35394]: 2024-05-03 18:38:55.006199946 +0000 UTC m=+0.118963216 container create 5dbb8da288720641a1a2cf3f69b36e77246af4204687fd3dd426a3d2c60ac87e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-z, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240430, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-05-03T18:38:55.253 INFO:journalctl@ceph.mgr.z.smithi113.stdout:May 03 18:38:55 smithi113 podman[35394]: 2024-05-03 18:38:55.200612435 +0000 UTC m=+0.313375646 container init 5dbb8da288720641a1a2cf3f69b36e77246af4204687fd3dd426a3d2c60ac87e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-z, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240430, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD) 2024-05-03T18:38:55.253 INFO:journalctl@ceph.mgr.z.smithi113.stdout:May 03 18:38:55 smithi113 podman[35394]: 2024-05-03 18:38:55.203164679 +0000 UTC m=+0.315927893 container start 5dbb8da288720641a1a2cf3f69b36e77246af4204687fd3dd426a3d2c60ac87e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-z, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240430, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-03T18:38:55.253 INFO:journalctl@ceph.mgr.z.smithi113.stdout:May 03 18:38:55 smithi113 ceph-mgr[35412]: -- 172.21.15.113:0/3088847390 <== mon.1 v2:172.21.15.113:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5604be21c5a0 con 0x5604be1f1000 2024-05-03T18:38:55.253 INFO:journalctl@ceph.mgr.z.smithi113.stdout:May 03 18:38:55 smithi113 bash[35394]: 5dbb8da288720641a1a2cf3f69b36e77246af4204687fd3dd426a3d2c60ac87e 2024-05-03T18:38:55.366 INFO:teuthology.orchestra.run.smithi032.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-03T18:38:55.367 INFO:teuthology.orchestra.run.smithi032.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-03T18:38:55.367 INFO:teuthology.orchestra.run.smithi032.stderr: stderr: 10+0 records in 2024-05-03T18:38:55.367 INFO:teuthology.orchestra.run.smithi032.stderr:10+0 records out 2024-05-03T18:38:55.367 INFO:teuthology.orchestra.run.smithi032.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00869882 s, 1.2 GB/s 2024-05-03T18:38:55.367 INFO:teuthology.orchestra.run.smithi032.stderr:--> Zapping successful for: 2024-05-03T18:38:55.538 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:38:55 smithi113 ceph-mon[33540]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:38:55.538 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:38:55 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:55.538 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:38:55 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:55.538 INFO:journalctl@ceph.mgr.z.smithi113.stdout:May 03 18:38:55 smithi113 systemd[1]: Started Ceph mgr.z for f500455e-097b-11ef-bc97-c7b262605968. 2024-05-03T18:38:55.539 INFO:journalctl@ceph.mgr.z.smithi113.stdout:May 03 18:38:55 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-z[35408]: 2024-05-03T18:38:55.356+0000 7fbda12b6200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-03T18:38:55.539 INFO:journalctl@ceph.mgr.z.smithi113.stdout:May 03 18:38:55 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-z[35408]: 2024-05-03T18:38:55.537+0000 7fbda12b6200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-03T18:38:55.747 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:55 smithi045 ceph-mon[33299]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:38:55.747 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:55 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:55.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:55 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:55.760 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:55 smithi032 ceph-mon[29763]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:38:55.760 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:55 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:55.761 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:55 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:55.800 INFO:journalctl@ceph.mgr.z.smithi113.stdout:May 03 18:38:55 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-z[35408]: 2024-05-03T18:38:55.639+0000 7fbda12b6200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-03T18:38:56.002 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f500455e-097b-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi032:vg_nvme/lv_4 2024-05-03T18:38:56.248 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:38:56.301 INFO:journalctl@ceph.mgr.z.smithi113.stdout:May 03 18:38:55 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-z[35408]: 2024-05-03T18:38:55.946+0000 7fbda12b6200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-03T18:38:56.301 INFO:journalctl@ceph.mgr.z.smithi113.stdout:May 03 18:38:56 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-z[35408]: 2024-05-03T18:38:56.029+0000 7fbda12b6200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-03T18:38:56.597 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:56 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:56.597 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:56 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-03T18:38:56.598 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:56 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-03T18:38:56.598 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:56 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-03T18:38:56.598 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:56 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:38:56.598 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:56 smithi032 ceph-mon[29763]: Deploying daemon mgr.y on smithi045 2024-05-03T18:38:56.620 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:38:56 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:56.620 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:38:56 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-03T18:38:56.620 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:38:56 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-03T18:38:56.620 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:38:56 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-03T18:38:56.620 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:38:56 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:38:56.620 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:38:56 smithi113 ceph-mon[33540]: Deploying daemon mgr.y on smithi045 2024-05-03T18:38:56.747 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:56 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:56.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:56 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-03T18:38:56.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:56 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-03T18:38:56.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:56 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-03T18:38:56.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:56 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:38:56.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:56 smithi045 ceph-mon[33299]: Deploying daemon mgr.y on smithi045 2024-05-03T18:38:57.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:57 smithi045 ceph-mon[33299]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:38:57.498 INFO:journalctl@ceph.mgr.y.smithi045.stdout:May 03 18:38:57 smithi045 systemd[1]: Starting Ceph mgr.y for f500455e-097b-11ef-bc97-c7b262605968... 2024-05-03T18:38:57.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:38:57 smithi113 ceph-mon[33540]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:38:57.551 INFO:journalctl@ceph.mgr.z.smithi113.stdout:May 03 18:38:57 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-z[35408]: 2024-05-03T18:38:57.240+0000 7fbda12b6200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-03T18:38:57.551 INFO:journalctl@ceph.mgr.z.smithi113.stdout:May 03 18:38:57 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-z[35408]: 2024-05-03T18:38:57.411+0000 7fbda12b6200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-03T18:38:57.670 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:57 smithi032 ceph-mon[29763]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:38:57.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:57.784+0000 7fa9c88b4700 1 -- 172.21.15.32:0/3216309655 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9c4103fc0 msgr2=0x7fa9c410a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:38:57.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:57.784+0000 7fa9c88b4700 1 --2- 172.21.15.32:0/3216309655 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9c4103fc0 0x7fa9c410a870 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fa9bc00b0d0 tx=0x7fa9bc00b3e0 comp rx=0 tx=0).stop 2024-05-03T18:38:57.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:57.785+0000 7fa9c88b4700 1 -- 172.21.15.32:0/3216309655 shutdown_connections 2024-05-03T18:38:57.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:57.785+0000 7fa9c88b4700 1 --2- 172.21.15.32:0/3216309655 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9c4103fc0 0x7fa9c410a870 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:57.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:57.785+0000 7fa9c88b4700 1 --2- 172.21.15.32:0/3216309655 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa9c4102420 0x7fa9c4102840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:57.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:57.785+0000 7fa9c88b4700 1 --2- 172.21.15.32:0/3216309655 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa9c4103620 0x7fa9c4103a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:57.786 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:57.785+0000 7fa9c88b4700 1 -- 172.21.15.32:0/3216309655 >> 172.21.15.32:0/3216309655 conn(0x7fa9c40fd9c0 msgr2=0x7fa9c40ffe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:38:57.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:57.785+0000 7fa9c88b4700 1 -- 172.21.15.32:0/3216309655 shutdown_connections 2024-05-03T18:38:57.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:57.785+0000 7fa9c88b4700 1 -- 172.21.15.32:0/3216309655 wait complete. 2024-05-03T18:38:57.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:57.785+0000 7fa9c88b4700 1 Processor -- start 2024-05-03T18:38:57.787 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:57.785+0000 7fa9c88b4700 1 -- start start 2024-05-03T18:38:57.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:57.786+0000 7fa9c88b4700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa9c4102420 0x7fa9c41063b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:38:57.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:57.786+0000 7fa9c88b4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9c4103620 0x7fa9c4104a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:38:57.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:57.786+0000 7fa9c88b4700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa9c4103fc0 0x7fa9c4104f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:38:57.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:57.786+0000 7fa9c88b4700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c410de90 con 0x7fa9c4103620 2024-05-03T18:38:57.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:57.786+0000 7fa9c88b4700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c410dd10 con 0x7fa9c4102420 2024-05-03T18:38:57.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:57.786+0000 7fa9c88b4700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c410e010 con 0x7fa9c4103fc0 2024-05-03T18:38:57.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:57.786+0000 7fa9c1d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9c4103620 0x7fa9c4104a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:38:57.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:57.786+0000 7fa9c1d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9c4103620 0x7fa9c4104a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41448/0 (socket says 172.21.15.32:41448) 2024-05-03T18:38:57.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:57.786+0000 7fa9c1d9b700 1 -- 172.21.15.32:0/3985802469 learned_addr learned my addr 172.21.15.32:0/3985802469 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:38:57.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:57.786+0000 7fa9c259c700 1 --2- 172.21.15.32:0/3985802469 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa9c4102420 0x7fa9c41063b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:38:57.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:57.786+0000 7fa9c2d9d700 1 --2- 172.21.15.32:0/3985802469 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa9c4103fc0 0x7fa9c4104f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:38:57.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:57.786+0000 7fa9c1d9b700 1 -- 172.21.15.32:0/3985802469 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa9c4103fc0 msgr2=0x7fa9c4104f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:38:57.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:57.786+0000 7fa9c1d9b700 1 --2- 172.21.15.32:0/3985802469 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa9c4103fc0 0x7fa9c4104f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:57.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:57.786+0000 7fa9c1d9b700 1 -- 172.21.15.32:0/3985802469 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa9c4102420 msgr2=0x7fa9c41063b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:38:57.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:57.786+0000 7fa9c1d9b700 1 --2- 172.21.15.32:0/3985802469 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa9c4102420 0x7fa9c41063b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:38:57.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:57.786+0000 7fa9c1d9b700 1 -- 172.21.15.32:0/3985802469 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9c41057f0 con 0x7fa9c4103620 2024-05-03T18:38:57.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:57.786+0000 7fa9c1d9b700 1 --2- 172.21.15.32:0/3985802469 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9c4103620 0x7fa9c4104a00 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fa9b800d950 tx=0x7fa9b800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:38:57.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:57.786+0000 7fa9b37fe700 1 -- 172.21.15.32:0/3985802469 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9b8016070 con 0x7fa9c4103620 2024-05-03T18:38:57.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:57.787+0000 7fa9c88b4700 1 -- 172.21.15.32:0/3985802469 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9c41a3760 con 0x7fa9c4103620 2024-05-03T18:38:57.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:57.787+0000 7fa9b37fe700 1 -- 172.21.15.32:0/3985802469 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9b800fd40 con 0x7fa9c4103620 2024-05-03T18:38:57.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:57.787+0000 7fa9b37fe700 1 -- 172.21.15.32:0/3985802469 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9b800e3b0 con 0x7fa9c4103620 2024-05-03T18:38:57.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:57.787+0000 7fa9c88b4700 1 -- 172.21.15.32:0/3985802469 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9c41a3c50 con 0x7fa9c4103620 2024-05-03T18:38:57.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:57.788+0000 7fa9b37fe700 1 -- 172.21.15.32:0/3985802469 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7fa9b8019070 con 0x7fa9c4103620 2024-05-03T18:38:57.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:57.788+0000 7fa9c88b4700 1 -- 172.21.15.32:0/3985802469 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9c406a190 con 0x7fa9c4103620 2024-05-03T18:38:57.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:57.788+0000 7fa9b37fe700 1 --2- 172.21.15.32:0/3985802469 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fa9ac03e750 0x7fa9ac040c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:38:57.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:57.788+0000 7fa9b37fe700 1 -- 172.21.15.32:0/3985802469 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fa9b8052850 con 0x7fa9c4103620 2024-05-03T18:38:57.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:57.788+0000 7fa9c259c700 1 --2- 172.21.15.32:0/3985802469 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fa9ac03e750 0x7fa9ac040c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:38:57.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:57.789+0000 7fa9c259c700 1 --2- 172.21.15.32:0/3985802469 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fa9ac03e750 0x7fa9ac040c10 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fa9c4103480 tx=0x7fa9b400a3b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:38:57.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:57.793+0000 7fa9b37fe700 1 -- 172.21.15.32:0/3985802469 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fa9b8010460 con 0x7fa9c4103620 2024-05-03T18:38:57.861 INFO:journalctl@ceph.mgr.y.smithi045.stdout:May 03 18:38:57 smithi045 podman[34657]: 2024-05-03 18:38:57.605491155 +0000 UTC m=+0.113598931 container create 57afcca919d642837a6928130b4ac0371edc1da73a012565916cb6afaf3579a2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-y, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240430, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-03T18:38:57.861 INFO:journalctl@ceph.mgr.y.smithi045.stdout:May 03 18:38:57 smithi045 podman[34657]: 2024-05-03 18:38:57.510525823 +0000 UTC m=+0.018633571 image pull ee4d846202496ba6096593e8e8becc0700a5eb7ebd95536031187f5f337d4012 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd 2024-05-03T18:38:57.861 INFO:journalctl@ceph.mgr.y.smithi045.stdout:May 03 18:38:57 smithi045 podman[34657]: 2024-05-03 18:38:57.833093424 +0000 UTC m=+0.341201159 container init 57afcca919d642837a6928130b4ac0371edc1da73a012565916cb6afaf3579a2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-y, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.build-date=20240430) 2024-05-03T18:38:57.861 INFO:journalctl@ceph.mgr.y.smithi045.stdout:May 03 18:38:57 smithi045 podman[34657]: 2024-05-03 18:38:57.835863546 +0000 UTC m=+0.343971286 container start 57afcca919d642837a6928130b4ac0371edc1da73a012565916cb6afaf3579a2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-y, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240430) 2024-05-03T18:38:57.862 INFO:journalctl@ceph.mgr.y.smithi045.stdout:May 03 18:38:57 smithi045 ceph-mgr[34676]: -- 172.21.15.45:0/1649723990 <== mon.2 v2:172.21.15.45:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56328ca965a0 con 0x56328ca6a800 2024-05-03T18:38:57.864 INFO:journalctl@ceph.mgr.z.smithi113.stdout:May 03 18:38:57 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-z[35408]: 2024-05-03T18:38:57.582+0000 7fbda12b6200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-03T18:38:57.864 INFO:journalctl@ceph.mgr.z.smithi113.stdout:May 03 18:38:57 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-z[35408]: 2024-05-03T18:38:57.863+0000 7fbda12b6200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-03T18:38:58.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:38:58.020+0000 7fa9c88b4700 1 -- 172.21.15.32:0/3985802469 --> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi032:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fa9c40ffac0 con 0x7fa9ac03e750 2024-05-03T18:38:58.155 INFO:journalctl@ceph.mgr.z.smithi113.stdout:May 03 18:38:57 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-z[35408]: 2024-05-03T18:38:57.954+0000 7fbda12b6200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-03T18:38:58.198 INFO:journalctl@ceph.mgr.y.smithi045.stdout:May 03 18:38:57 smithi045 bash[34657]: 57afcca919d642837a6928130b4ac0371edc1da73a012565916cb6afaf3579a2 2024-05-03T18:38:58.198 INFO:journalctl@ceph.mgr.y.smithi045.stdout:May 03 18:38:57 smithi045 systemd[1]: Started Ceph mgr.y for f500455e-097b-11ef-bc97-c7b262605968. 2024-05-03T18:38:58.198 INFO:journalctl@ceph.mgr.y.smithi045.stdout:May 03 18:38:57 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-y[34672]: 2024-05-03T18:38:57.996+0000 7f695e285200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-03T18:38:58.198 INFO:journalctl@ceph.mgr.y.smithi045.stdout:May 03 18:38:58 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-y[34672]: 2024-05-03T18:38:58.196+0000 7f695e285200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-03T18:38:58.497 INFO:journalctl@ceph.mgr.y.smithi045.stdout:May 03 18:38:58 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-y[34672]: 2024-05-03T18:38:58.310+0000 7f695e285200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-03T18:38:58.551 INFO:journalctl@ceph.mgr.z.smithi113.stdout:May 03 18:38:58 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-z[35408]: 2024-05-03T18:38:58.154+0000 7fbda12b6200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-03T18:38:58.551 INFO:journalctl@ceph.mgr.z.smithi113.stdout:May 03 18:38:58 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-z[35408]: 2024-05-03T18:38:58.354+0000 7fbda12b6200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-03T18:38:58.990 INFO:journalctl@ceph.mgr.y.smithi045.stdout:May 03 18:38:58 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-y[34672]: 2024-05-03T18:38:58.660+0000 7f695e285200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-03T18:38:58.990 INFO:journalctl@ceph.mgr.y.smithi045.stdout:May 03 18:38:58 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-y[34672]: 2024-05-03T18:38:58.771+0000 7f695e285200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-03T18:38:59.204 INFO:journalctl@ceph.mgr.z.smithi113.stdout:May 03 18:38:58 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-z[35408]: 2024-05-03T18:38:58.977+0000 7fbda12b6200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-03T18:38:59.204 INFO:journalctl@ceph.mgr.z.smithi113.stdout:May 03 18:38:59 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-z[35408]: 2024-05-03T18:38:59.203+0000 7fbda12b6200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-03T18:38:59.204 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:38:58 smithi113 ceph-mon[33540]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:38:59.204 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:38:58 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:59.204 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:38:58 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:59.204 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:38:58 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:59.204 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:38:58 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:59.205 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:38:58 smithi113 ceph-mon[33540]: from='client.14217 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi032:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:38:59.205 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:38:58 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T18:38:59.205 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:38:58 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T18:38:59.205 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:38:58 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:38:59.205 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:38:58 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:38:59.209 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:58 smithi032 ceph-mon[29763]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:38:59.209 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:58 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:59.209 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:58 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:59.209 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:58 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:59.210 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:58 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:59.210 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:58 smithi032 ceph-mon[29763]: from='client.14217 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi032:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:38:59.210 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:58 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T18:38:59.210 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:58 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T18:38:59.210 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:58 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:38:59.210 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:38:58 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:38:59.247 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:58 smithi045 ceph-mon[33299]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:38:59.247 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:58 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:59.248 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:58 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:59.248 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:58 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:59.248 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:58 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:38:59.248 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:58 smithi045 ceph-mon[33299]: from='client.14217 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi032:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:38:59.248 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:58 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T18:38:59.248 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:58 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T18:38:59.248 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:58 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:38:59.248 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:38:58 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:39:00.051 INFO:journalctl@ceph.mgr.z.smithi113.stdout:May 03 18:38:59 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-z[35408]: 2024-05-03T18:38:59.632+0000 7fbda12b6200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-03T18:39:00.051 INFO:journalctl@ceph.mgr.z.smithi113.stdout:May 03 18:38:59 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-z[35408]: 2024-05-03T18:38:59.730+0000 7fbda12b6200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-03T18:39:00.384 INFO:journalctl@ceph.mgr.z.smithi113.stdout:May 03 18:39:00 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-z[35408]: 2024-05-03T18:39:00.091+0000 7fbda12b6200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-03T18:39:00.384 INFO:journalctl@ceph.mgr.y.smithi045.stdout:May 03 18:39:00 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-y[34672]: 2024-05-03T18:39:00.100+0000 7f695e285200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-03T18:39:00.385 INFO:journalctl@ceph.mgr.y.smithi045.stdout:May 03 18:39:00 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-y[34672]: 2024-05-03T18:39:00.276+0000 7f695e285200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-03T18:39:00.747 INFO:journalctl@ceph.mgr.y.smithi045.stdout:May 03 18:39:00 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-y[34672]: 2024-05-03T18:39:00.453+0000 7f695e285200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-03T18:39:00.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:00 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:39:00.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:00 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:39:00.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:00 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:39:00.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:00 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:39:00.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:00 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:39:00.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:00 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:39:00.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:00 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:39:00.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:00 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-03T18:39:00.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:00 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-03T18:39:00.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:00 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:39:00.766 INFO:journalctl@ceph.mgr.z.smithi113.stdout:May 03 18:39:00 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-z[35408]: 2024-05-03T18:39:00.439+0000 7fbda12b6200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-03T18:39:00.766 INFO:journalctl@ceph.mgr.z.smithi113.stdout:May 03 18:39:00 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-z[35408]: 2024-05-03T18:39:00.529+0000 7fbda12b6200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-03T18:39:00.766 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:00 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:39:00.766 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:00 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:39:00.767 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:00 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:39:00.767 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:00 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:39:00.767 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:00 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:39:00.767 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:00 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:39:00.767 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:00 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:39:00.767 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:00 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-03T18:39:00.767 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:00 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-03T18:39:00.767 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:00 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:39:00.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:00 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:39:00.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:00 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:39:00.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:00 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:39:00.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:00 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:39:00.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:00 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:39:00.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:00 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:39:00.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:00 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:39:00.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:00 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-03T18:39:00.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:00 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-03T18:39:00.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:00 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:39:01.050 INFO:journalctl@ceph.mgr.z.smithi113.stdout:May 03 18:39:00 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-z[35408]: 2024-05-03T18:39:00.765+0000 7fbda12b6200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-03T18:39:01.051 INFO:journalctl@ceph.mgr.z.smithi113.stdout:May 03 18:39:00 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-z[35408]: 2024-05-03T18:39:00.895+0000 7fbda12b6200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-03T18:39:01.058 INFO:journalctl@ceph.mgr.y.smithi045.stdout:May 03 18:39:00 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-y[34672]: 2024-05-03T18:39:00.753+0000 7f695e285200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-03T18:39:01.058 INFO:journalctl@ceph.mgr.y.smithi045.stdout:May 03 18:39:00 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-y[34672]: 2024-05-03T18:39:00.848+0000 7f695e285200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-03T18:39:01.058 INFO:journalctl@ceph.mgr.y.smithi045.stdout:May 03 18:39:01 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-y[34672]: 2024-05-03T18:39:01.057+0000 7f695e285200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-03T18:39:01.494 INFO:journalctl@ceph.mgr.y.smithi045.stdout:May 03 18:39:01 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-y[34672]: 2024-05-03T18:39:01.256+0000 7f695e285200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-03T18:39:01.495 INFO:journalctl@ceph.mgr.z.smithi113.stdout:May 03 18:39:01 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-z[35408]: 2024-05-03T18:39:01.194+0000 7fbda12b6200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-03T18:39:01.495 INFO:journalctl@ceph.mgr.z.smithi113.stdout:May 03 18:39:01 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-z[35408]: 2024-05-03T18:39:01.306+0000 7fbda12b6200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-03T18:39:01.495 INFO:journalctl@ceph.mgr.z.smithi113.stdout:May 03 18:39:01 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-z[35408]: 2024-05-03T18:39:01.395+0000 7fbda12b6200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-03T18:39:01.747 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:01 smithi045 ceph-mon[33299]: Reconfiguring mgr.x (unknown last config time)... 2024-05-03T18:39:01.747 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:01 smithi045 ceph-mon[33299]: Reconfiguring daemon mgr.x on smithi032 2024-05-03T18:39:01.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:01 smithi045 ceph-mon[33299]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:39:01.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:01 smithi045 ceph-mon[33299]: from='client.? 172.21.15.32:0/3307472233' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0430c3c0-2c50-471c-9c74-2869a0627c74"}]: dispatch 2024-05-03T18:39:01.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:01 smithi045 ceph-mon[33299]: from='client.? 172.21.15.32:0/3307472233' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0430c3c0-2c50-471c-9c74-2869a0627c74"}]': finished 2024-05-03T18:39:01.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:01 smithi045 ceph-mon[33299]: osdmap e5: 1 total, 0 up, 1 in 2024-05-03T18:39:01.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:01 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-03T18:39:01.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:01 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:39:01.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:01 smithi045 ceph-mon[33299]: from='client.? 172.21.15.32:0/1372877621' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T18:39:01.800 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:01 smithi113 ceph-mon[33540]: Reconfiguring mgr.x (unknown last config time)... 2024-05-03T18:39:01.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:01 smithi113 ceph-mon[33540]: Reconfiguring daemon mgr.x on smithi032 2024-05-03T18:39:01.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:01 smithi113 ceph-mon[33540]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:39:01.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:01 smithi113 ceph-mon[33540]: from='client.? 172.21.15.32:0/3307472233' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0430c3c0-2c50-471c-9c74-2869a0627c74"}]: dispatch 2024-05-03T18:39:01.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:01 smithi113 ceph-mon[33540]: from='client.? 172.21.15.32:0/3307472233' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0430c3c0-2c50-471c-9c74-2869a0627c74"}]': finished 2024-05-03T18:39:01.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:01 smithi113 ceph-mon[33540]: osdmap e5: 1 total, 0 up, 1 in 2024-05-03T18:39:01.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:01 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-03T18:39:01.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:01 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:39:01.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:01 smithi113 ceph-mon[33540]: from='client.? 172.21.15.32:0/1372877621' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T18:39:01.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:01 smithi032 ceph-mon[29763]: Reconfiguring mgr.x (unknown last config time)... 2024-05-03T18:39:01.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:01 smithi032 ceph-mon[29763]: Reconfiguring daemon mgr.x on smithi032 2024-05-03T18:39:01.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:01 smithi032 ceph-mon[29763]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:39:01.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:01 smithi032 ceph-mon[29763]: from='client.? 172.21.15.32:0/3307472233' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0430c3c0-2c50-471c-9c74-2869a0627c74"}]: dispatch 2024-05-03T18:39:01.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:01 smithi032 ceph-mon[29763]: from='client.? 172.21.15.32:0/3307472233' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0430c3c0-2c50-471c-9c74-2869a0627c74"}]': finished 2024-05-03T18:39:01.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:01 smithi032 ceph-mon[29763]: osdmap e5: 1 total, 0 up, 1 in 2024-05-03T18:39:01.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:01 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-03T18:39:01.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:01 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:39:01.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:01 smithi032 ceph-mon[29763]: from='client.? 172.21.15.32:0/1372877621' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T18:39:02.247 INFO:journalctl@ceph.mgr.y.smithi045.stdout:May 03 18:39:01 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-y[34672]: 2024-05-03T18:39:01.884+0000 7f695e285200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-03T18:39:02.247 INFO:journalctl@ceph.mgr.y.smithi045.stdout:May 03 18:39:02 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-y[34672]: 2024-05-03T18:39:02.111+0000 7f695e285200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-03T18:39:02.300 INFO:journalctl@ceph.mgr.z.smithi113.stdout:May 03 18:39:01 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-z[35408]: 2024-05-03T18:39:01.983+0000 7fbda12b6200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-03T18:39:02.301 INFO:journalctl@ceph.mgr.z.smithi113.stdout:May 03 18:39:02 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-z[35408]: 2024-05-03T18:39:02.194+0000 7fbda12b6200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-03T18:39:02.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:39:02.502+0000 7fa9b37fe700 1 -- 172.21.15.32:0/3985802469 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mgrmap(e 14) ==== 99937+0+0 (secure 0 0 0) 0x7fa9b8019c80 con 0x7fa9c4103620 2024-05-03T18:39:02.747 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:02 smithi045 ceph-mon[33299]: Standby manager daemon z started 2024-05-03T18:39:02.747 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:02 smithi045 ceph-mon[33299]: from='mgr.? 172.21.15.113:0/3862475952' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-03T18:39:02.747 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:02 smithi045 ceph-mon[33299]: from='mgr.? 172.21.15.113:0/3862475952' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-03T18:39:02.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:02 smithi045 ceph-mon[33299]: from='mgr.? 172.21.15.113:0/3862475952' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-03T18:39:02.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:02 smithi045 ceph-mon[33299]: from='mgr.? 172.21.15.113:0/3862475952' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-03T18:39:02.748 INFO:journalctl@ceph.mgr.y.smithi045.stdout:May 03 18:39:02 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-y[34672]: 2024-05-03T18:39:02.549+0000 7f695e285200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-03T18:39:02.748 INFO:journalctl@ceph.mgr.y.smithi045.stdout:May 03 18:39:02 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-y[34672]: 2024-05-03T18:39:02.648+0000 7f695e285200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-03T18:39:02.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:02 smithi113 ceph-mon[33540]: Standby manager daemon z started 2024-05-03T18:39:02.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:02 smithi113 ceph-mon[33540]: from='mgr.? 172.21.15.113:0/3862475952' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-03T18:39:02.802 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:02 smithi113 ceph-mon[33540]: from='mgr.? 172.21.15.113:0/3862475952' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-03T18:39:02.802 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:02 smithi113 ceph-mon[33540]: from='mgr.? 172.21.15.113:0/3862475952' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-03T18:39:02.802 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:02 smithi113 ceph-mon[33540]: from='mgr.? 172.21.15.113:0/3862475952' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-03T18:39:02.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:02 smithi032 ceph-mon[29763]: Standby manager daemon z started 2024-05-03T18:39:02.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:02 smithi032 ceph-mon[29763]: from='mgr.? 172.21.15.113:0/3862475952' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-03T18:39:02.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:02 smithi032 ceph-mon[29763]: from='mgr.? 172.21.15.113:0/3862475952' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-03T18:39:02.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:02 smithi032 ceph-mon[29763]: from='mgr.? 172.21.15.113:0/3862475952' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-03T18:39:02.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:02 smithi032 ceph-mon[29763]: from='mgr.? 172.21.15.113:0/3862475952' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-03T18:39:03.402 INFO:journalctl@ceph.mgr.y.smithi045.stdout:May 03 18:39:03 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-y[34672]: 2024-05-03T18:39:03.030+0000 7f695e285200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-03T18:39:03.402 INFO:journalctl@ceph.mgr.y.smithi045.stdout:May 03 18:39:03 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-y[34672]: 2024-05-03T18:39:03.400+0000 7f695e285200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-03T18:39:03.747 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:03 smithi045 ceph-mon[33299]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:39:03.747 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:03 smithi045 ceph-mon[33299]: mgrmap e14: x(active, since 76s), standbys: z 2024-05-03T18:39:03.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:03 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-03T18:39:03.748 INFO:journalctl@ceph.mgr.y.smithi045.stdout:May 03 18:39:03 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-y[34672]: 2024-05-03T18:39:03.495+0000 7f695e285200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-03T18:39:03.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:03 smithi113 ceph-mon[33540]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:39:03.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:03 smithi113 ceph-mon[33540]: mgrmap e14: x(active, since 76s), standbys: z 2024-05-03T18:39:03.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:03 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-03T18:39:03.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:03 smithi032 ceph-mon[29763]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:39:03.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:03 smithi032 ceph-mon[29763]: mgrmap e14: x(active, since 76s), standbys: z 2024-05-03T18:39:03.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:03 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-03T18:39:04.200 INFO:journalctl@ceph.mgr.y.smithi045.stdout:May 03 18:39:03 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-y[34672]: 2024-05-03T18:39:03.747+0000 7f695e285200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-03T18:39:04.200 INFO:journalctl@ceph.mgr.y.smithi045.stdout:May 03 18:39:03 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-y[34672]: 2024-05-03T18:39:03.885+0000 7f695e285200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-03T18:39:04.200 INFO:journalctl@ceph.mgr.y.smithi045.stdout:May 03 18:39:04 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-y[34672]: 2024-05-03T18:39:04.198+0000 7f695e285200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-03T18:39:04.497 INFO:journalctl@ceph.mgr.y.smithi045.stdout:May 03 18:39:04 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-y[34672]: 2024-05-03T18:39:04.318+0000 7f695e285200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-03T18:39:04.497 INFO:journalctl@ceph.mgr.y.smithi045.stdout:May 03 18:39:04 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-y[34672]: 2024-05-03T18:39:04.412+0000 7f695e285200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-03T18:39:05.497 INFO:journalctl@ceph.mgr.y.smithi045.stdout:May 03 18:39:05 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-y[34672]: 2024-05-03T18:39:05.039+0000 7f695e285200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-03T18:39:05.498 INFO:journalctl@ceph.mgr.y.smithi045.stdout:May 03 18:39:05 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-y[34672]: 2024-05-03T18:39:05.262+0000 7f695e285200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-03T18:39:05.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:39:05.515+0000 7fa9b37fe700 1 -- 172.21.15.32:0/3985802469 <== mon.0 v2:172.21.15.32:3300/0 8 ==== mgrmap(e 15) ==== 149596+0+0 (secure 0 0 0) 0x7fa9b80669f0 con 0x7fa9c4103620 2024-05-03T18:39:05.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:05 smithi113 ceph-mon[33540]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:39:05.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:05 smithi113 ceph-mon[33540]: Standby manager daemon y started 2024-05-03T18:39:05.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:05 smithi113 ceph-mon[33540]: from='mgr.? 172.21.15.45:0/2521052571' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-03T18:39:05.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:05 smithi113 ceph-mon[33540]: from='mgr.? 172.21.15.45:0/2521052571' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-03T18:39:05.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:05 smithi113 ceph-mon[33540]: from='mgr.? 172.21.15.45:0/2521052571' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-03T18:39:05.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:05 smithi113 ceph-mon[33540]: from='mgr.? 172.21.15.45:0/2521052571' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-03T18:39:05.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:05 smithi032 ceph-mon[29763]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:39:05.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:05 smithi032 ceph-mon[29763]: Standby manager daemon y started 2024-05-03T18:39:05.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:05 smithi032 ceph-mon[29763]: from='mgr.? 172.21.15.45:0/2521052571' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-03T18:39:05.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:05 smithi032 ceph-mon[29763]: from='mgr.? 172.21.15.45:0/2521052571' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-03T18:39:05.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:05 smithi032 ceph-mon[29763]: from='mgr.? 172.21.15.45:0/2521052571' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-03T18:39:05.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:05 smithi032 ceph-mon[29763]: from='mgr.? 172.21.15.45:0/2521052571' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-03T18:39:05.997 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:05 smithi045 ceph-mon[33299]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:39:05.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:05 smithi045 ceph-mon[33299]: Standby manager daemon y started 2024-05-03T18:39:05.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:05 smithi045 ceph-mon[33299]: from='mgr.? 172.21.15.45:0/2521052571' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-03T18:39:05.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:05 smithi045 ceph-mon[33299]: from='mgr.? 172.21.15.45:0/2521052571' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-03T18:39:05.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:05 smithi045 ceph-mon[33299]: from='mgr.? 172.21.15.45:0/2521052571' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-03T18:39:05.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:05 smithi045 ceph-mon[33299]: from='mgr.? 172.21.15.45:0/2521052571' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-03T18:39:06.660 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:06 smithi032 ceph-mon[29763]: mgrmap e15: x(active, since 79s), standbys: z, y 2024-05-03T18:39:06.660 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:06 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-03T18:39:06.661 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:06 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:39:06.661 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:06 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:39:06.661 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:06 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:39:06.661 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:06 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:39:06.661 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:06 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:39:06.661 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:06 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:39:06.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:06 smithi113 ceph-mon[33540]: mgrmap e15: x(active, since 79s), standbys: z, y 2024-05-03T18:39:06.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:06 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-03T18:39:06.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:06 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:39:06.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:06 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:39:06.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:06 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:39:06.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:06 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:39:06.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:06 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:39:06.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:06 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:39:06.997 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:06 smithi045 ceph-mon[33299]: mgrmap e15: x(active, since 79s), standbys: z, y 2024-05-03T18:39:06.997 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:06 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-03T18:39:06.997 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:06 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:39:06.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:06 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:39:06.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:06 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:39:06.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:06 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:39:06.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:06 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:39:06.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:06 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:39:07.800 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:07 smithi113 ceph-mon[33540]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:39:07.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:07 smithi032 ceph-mon[29763]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:39:07.997 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:07 smithi045 ceph-mon[33299]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:39:08.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:08 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-03T18:39:08.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:08 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:39:08.809 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:08 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-03T18:39:08.810 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:08 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:39:08.827 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:08 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-03T18:39:08.827 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:08 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:39:09.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:09 smithi045 ceph-mon[33299]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:39:09.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:09 smithi045 ceph-mon[33299]: Deploying daemon osd.0 on smithi032 2024-05-03T18:39:09.800 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:09 smithi113 ceph-mon[33540]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:39:09.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:09 smithi113 ceph-mon[33540]: Deploying daemon osd.0 on smithi032 2024-05-03T18:39:09.813 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:09 smithi032 ceph-mon[29763]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:39:09.813 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:09 smithi032 ceph-mon[29763]: Deploying daemon osd.0 on smithi032 2024-05-03T18:39:11.782 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:11 smithi032 ceph-mon[29763]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:39:11.800 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:11 smithi113 ceph-mon[33540]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:39:11.997 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:11 smithi045 ceph-mon[33299]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:39:13.800 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:13 smithi113 ceph-mon[33540]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:39:13.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:13 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:39:13.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:13 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:39:13.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:13 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:39:13.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:13 smithi032 ceph-mon[29763]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:39:13.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:13 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:39:13.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:13 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:39:13.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:13 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:39:13.997 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:13 smithi045 ceph-mon[33299]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:39:13.997 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:13 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:39:13.997 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:13 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:39:13.997 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:13 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:39:14.800 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:14 smithi113 ceph-mon[33540]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:39:14.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:14 smithi032 ceph-mon[29763]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:39:14.997 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:14 smithi045 ceph-mon[33299]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:39:16.954 INFO:teuthology.orchestra.run.smithi032.stdout:Created osd(s) 0 on host 'smithi032' 2024-05-03T18:39:16.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:39:16.954+0000 7fa9b37fe700 1 -- 172.21.15.32:0/3985802469 <== mgr.14152 v2:172.21.15.32:6800/1796547904 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fa9c40ffac0 con 0x7fa9ac03e750 2024-05-03T18:39:16.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:39:16.956+0000 7fa9c88b4700 1 -- 172.21.15.32:0/3985802469 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fa9ac03e750 msgr2=0x7fa9ac040c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:39:16.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:39:16.956+0000 7fa9c88b4700 1 --2- 172.21.15.32:0/3985802469 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fa9ac03e750 0x7fa9ac040c10 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fa9c4103480 tx=0x7fa9b400a3b0 comp rx=0 tx=0).stop 2024-05-03T18:39:16.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:39:16.956+0000 7fa9c88b4700 1 -- 172.21.15.32:0/3985802469 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9c4103620 msgr2=0x7fa9c4104a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:39:16.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:39:16.956+0000 7fa9c88b4700 1 --2- 172.21.15.32:0/3985802469 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9c4103620 0x7fa9c4104a00 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fa9b800d950 tx=0x7fa9b800dc60 comp rx=0 tx=0).stop 2024-05-03T18:39:16.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:39:16.956+0000 7fa9c88b4700 1 -- 172.21.15.32:0/3985802469 shutdown_connections 2024-05-03T18:39:16.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:39:16.956+0000 7fa9c88b4700 1 --2- 172.21.15.32:0/3985802469 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fa9ac03e750 0x7fa9ac040c10 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:16.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:39:16.956+0000 7fa9c88b4700 1 --2- 172.21.15.32:0/3985802469 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa9c4103fc0 0x7fa9c4104f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:16.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:39:16.956+0000 7fa9c88b4700 1 --2- 172.21.15.32:0/3985802469 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa9c4102420 0x7fa9c41063b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:16.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:39:16.956+0000 7fa9c88b4700 1 --2- 172.21.15.32:0/3985802469 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9c4103620 0x7fa9c4104a00 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:16.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:39:16.956+0000 7fa9c88b4700 1 -- 172.21.15.32:0/3985802469 >> 172.21.15.32:0/3985802469 conn(0x7fa9c40fd9c0 msgr2=0x7fa9c40ff3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:39:16.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:39:16.956+0000 7fa9c88b4700 1 -- 172.21.15.32:0/3985802469 shutdown_connections 2024-05-03T18:39:16.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:39:16.956+0000 7fa9c88b4700 1 -- 172.21.15.32:0/3985802469 wait complete. 2024-05-03T18:39:17.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:17 smithi032 ceph-mon[29763]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:39:17.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:17 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:39:17.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:17 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:39:17.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:17 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:39:17.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:17 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:39:17.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:17 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:39:17.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:17 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:39:17.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:17 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:39:17.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:17 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:39:17.497 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:17 smithi045 ceph-mon[33299]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:39:17.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:17 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:39:17.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:17 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:39:17.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:17 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:39:17.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:17 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:39:17.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:17 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:39:17.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:17 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:39:17.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:17 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:39:17.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:17 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:39:17.541 DEBUG:teuthology.orchestra.run.smithi032:osd.0> sudo journalctl -f -n 0 -u ceph-f500455e-097b-11ef-bc97-c7b262605968@osd.0.service 2024-05-03T18:39:17.544 INFO:tasks.cephadm:Deploying osd.1 on smithi045 with /dev/vg_nvme/lv_4... 2024-05-03T18:39:17.544 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f500455e-097b-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-03T18:39:17.550 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:17 smithi113 ceph-mon[33540]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:39:17.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:17 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:39:17.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:17 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:39:17.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:17 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:39:17.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:17 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:39:17.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:17 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:39:17.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:17 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:39:17.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:17 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:39:17.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:17 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:39:17.799 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/mon.b/config 2024-05-03T18:39:18.171 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:18 smithi045 ceph-mon[33299]: from='osd.0 [v2:172.21.15.32:6802/262324956,v1:172.21.15.32:6803/262324956]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-03T18:39:18.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:18 smithi032 ceph-mon[29763]: from='osd.0 [v2:172.21.15.32:6802/262324956,v1:172.21.15.32:6803/262324956]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-03T18:39:18.550 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:18 smithi113 ceph-mon[33540]: from='osd.0 [v2:172.21.15.32:6802/262324956,v1:172.21.15.32:6803/262324956]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-03T18:39:19.064 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:19 smithi032 ceph-mon[29763]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:39:19.064 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:19 smithi032 ceph-mon[29763]: from='osd.0 [v2:172.21.15.32:6802/262324956,v1:172.21.15.32:6803/262324956]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-03T18:39:19.064 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:19 smithi032 ceph-mon[29763]: osdmap e6: 1 total, 0 up, 1 in 2024-05-03T18:39:19.064 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:19 smithi032 ceph-mon[29763]: from='osd.0 [v2:172.21.15.32:6802/262324956,v1:172.21.15.32:6803/262324956]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi032", "root=default"]}]: dispatch 2024-05-03T18:39:19.064 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:19 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-03T18:39:19.497 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:19 smithi045 ceph-mon[33299]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:39:19.497 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:19 smithi045 ceph-mon[33299]: from='osd.0 [v2:172.21.15.32:6802/262324956,v1:172.21.15.32:6803/262324956]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-03T18:39:19.497 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:19 smithi045 ceph-mon[33299]: osdmap e6: 1 total, 0 up, 1 in 2024-05-03T18:39:19.497 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:19 smithi045 ceph-mon[33299]: from='osd.0 [v2:172.21.15.32:6802/262324956,v1:172.21.15.32:6803/262324956]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi032", "root=default"]}]: dispatch 2024-05-03T18:39:19.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:19 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-03T18:39:19.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:19 smithi113 ceph-mon[33540]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:39:19.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:19 smithi113 ceph-mon[33540]: from='osd.0 [v2:172.21.15.32:6802/262324956,v1:172.21.15.32:6803/262324956]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-03T18:39:19.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:19 smithi113 ceph-mon[33540]: osdmap e6: 1 total, 0 up, 1 in 2024-05-03T18:39:19.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:19 smithi113 ceph-mon[33540]: from='osd.0 [v2:172.21.15.32:6802/262324956,v1:172.21.15.32:6803/262324956]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi032", "root=default"]}]: dispatch 2024-05-03T18:39:19.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:19 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-03T18:39:19.606 INFO:teuthology.orchestra.run.smithi045.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-03T18:39:19.606 INFO:teuthology.orchestra.run.smithi045.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-03T18:39:19.606 INFO:teuthology.orchestra.run.smithi045.stderr: stderr: 10+0 records in 2024-05-03T18:39:19.606 INFO:teuthology.orchestra.run.smithi045.stderr:10+0 records out 2024-05-03T18:39:19.606 INFO:teuthology.orchestra.run.smithi045.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00848532 s, 1.2 GB/s 2024-05-03T18:39:19.606 INFO:teuthology.orchestra.run.smithi045.stderr:--> Zapping successful for: 2024-05-03T18:39:20.063 INFO:journalctl@ceph.osd.0.smithi032.stdout:May 03 18:39:19 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-0[42356]: 2024-05-03T18:39:19.802+0000 7f6e81d7e700 -1 osd.0 0 waiting for initial osdmap 2024-05-03T18:39:20.064 INFO:journalctl@ceph.osd.0.smithi032.stdout:May 03 18:39:19 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-0[42356]: 2024-05-03T18:39:19.806+0000 7f6e79368700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-03T18:39:20.241 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:20 smithi045 ceph-mon[33299]: from='osd.0 [v2:172.21.15.32:6802/262324956,v1:172.21.15.32:6803/262324956]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi032", "root=default"]}]': finished 2024-05-03T18:39:20.241 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:20 smithi045 ceph-mon[33299]: osdmap e7: 1 total, 0 up, 1 in 2024-05-03T18:39:20.241 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:20 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-03T18:39:20.241 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:20 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-03T18:39:20.242 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:20 smithi045 ceph-mon[33299]: from='osd.0 [v2:172.21.15.32:6802/262324956,v1:172.21.15.32:6803/262324956]' entity='osd.0' 2024-05-03T18:39:20.270 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f500455e-097b-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi045:vg_nvme/lv_4 2024-05-03T18:39:20.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:20 smithi032 ceph-mon[29763]: from='osd.0 [v2:172.21.15.32:6802/262324956,v1:172.21.15.32:6803/262324956]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi032", "root=default"]}]': finished 2024-05-03T18:39:20.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:20 smithi032 ceph-mon[29763]: osdmap e7: 1 total, 0 up, 1 in 2024-05-03T18:39:20.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:20 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-03T18:39:20.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:20 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-03T18:39:20.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:20 smithi032 ceph-mon[29763]: from='osd.0 [v2:172.21.15.32:6802/262324956,v1:172.21.15.32:6803/262324956]' entity='osd.0' 2024-05-03T18:39:20.521 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/mon.b/config 2024-05-03T18:39:20.550 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:20 smithi113 ceph-mon[33540]: from='osd.0 [v2:172.21.15.32:6802/262324956,v1:172.21.15.32:6803/262324956]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi032", "root=default"]}]': finished 2024-05-03T18:39:20.550 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:20 smithi113 ceph-mon[33540]: osdmap e7: 1 total, 0 up, 1 in 2024-05-03T18:39:20.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:20 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-03T18:39:20.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:20 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-03T18:39:20.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:20 smithi113 ceph-mon[33540]: from='osd.0 [v2:172.21.15.32:6802/262324956,v1:172.21.15.32:6803/262324956]' entity='osd.0' 2024-05-03T18:39:21.247 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:21 smithi045 ceph-mon[33299]: purged_snaps scrub starts 2024-05-03T18:39:21.248 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:21 smithi045 ceph-mon[33299]: purged_snaps scrub ok 2024-05-03T18:39:21.248 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:21 smithi045 ceph-mon[33299]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:39:21.248 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:21 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-03T18:39:21.248 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:21 smithi045 ceph-mon[33299]: Detected new or changed devices on smithi032 2024-05-03T18:39:21.248 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:21 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:39:21.248 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:21 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:39:21.248 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:21 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:39:21.248 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:21 smithi045 ceph-mon[33299]: Adjusting osd_memory_target on smithi032 to 17093M 2024-05-03T18:39:21.248 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:21 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:39:21.248 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:21 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:39:21.248 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:21 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:39:21.248 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:21 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:39:21.249 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:21 smithi045 ceph-mon[33299]: osd.0 [v2:172.21.15.32:6802/262324956,v1:172.21.15.32:6803/262324956] boot 2024-05-03T18:39:21.249 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:21 smithi045 ceph-mon[33299]: osdmap e8: 1 total, 1 up, 1 in 2024-05-03T18:39:21.249 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:21 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-03T18:39:21.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:21 smithi032 ceph-mon[29763]: purged_snaps scrub starts 2024-05-03T18:39:21.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:21 smithi032 ceph-mon[29763]: purged_snaps scrub ok 2024-05-03T18:39:21.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:21 smithi032 ceph-mon[29763]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:39:21.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:21 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-03T18:39:21.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:21 smithi032 ceph-mon[29763]: Detected new or changed devices on smithi032 2024-05-03T18:39:21.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:21 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:39:21.319 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:21 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:39:21.319 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:21 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:39:21.319 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:21 smithi032 ceph-mon[29763]: Adjusting osd_memory_target on smithi032 to 17093M 2024-05-03T18:39:21.319 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:21 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:39:21.319 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:21 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:39:21.319 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:21 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:39:21.319 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:21 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:39:21.319 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:21 smithi032 ceph-mon[29763]: osd.0 [v2:172.21.15.32:6802/262324956,v1:172.21.15.32:6803/262324956] boot 2024-05-03T18:39:21.319 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:21 smithi032 ceph-mon[29763]: osdmap e8: 1 total, 1 up, 1 in 2024-05-03T18:39:21.319 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:21 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-03T18:39:21.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:21 smithi113 ceph-mon[33540]: purged_snaps scrub starts 2024-05-03T18:39:21.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:21 smithi113 ceph-mon[33540]: purged_snaps scrub ok 2024-05-03T18:39:21.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:21 smithi113 ceph-mon[33540]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-03T18:39:21.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:21 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-03T18:39:21.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:21 smithi113 ceph-mon[33540]: Detected new or changed devices on smithi032 2024-05-03T18:39:21.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:21 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:39:21.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:21 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:39:21.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:21 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:39:21.552 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:21 smithi113 ceph-mon[33540]: Adjusting osd_memory_target on smithi032 to 17093M 2024-05-03T18:39:21.552 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:21 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:39:21.552 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:21 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:39:21.552 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:21 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:39:21.552 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:21 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:39:21.552 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:21 smithi113 ceph-mon[33540]: osd.0 [v2:172.21.15.32:6802/262324956,v1:172.21.15.32:6803/262324956] boot 2024-05-03T18:39:21.552 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:21 smithi113 ceph-mon[33540]: osdmap e8: 1 total, 1 up, 1 in 2024-05-03T18:39:21.552 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:21 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-03T18:39:22.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:39:22.159+0000 7f2574c2c700 1 -- 172.21.15.45:0/541019899 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2570103fc0 msgr2=0x7f257010a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:39:22.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:39:22.159+0000 7f2574c2c700 1 --2- 172.21.15.45:0/541019899 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2570103fc0 0x7f257010a870 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f25680099c0 tx=0x7f2568009cd0 comp rx=0 tx=0).stop 2024-05-03T18:39:22.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:39:22.159+0000 7f2574c2c700 1 -- 172.21.15.45:0/541019899 shutdown_connections 2024-05-03T18:39:22.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:39:22.159+0000 7f2574c2c700 1 --2- 172.21.15.45:0/541019899 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2570103fc0 0x7f257010a870 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:22.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:39:22.159+0000 7f2574c2c700 1 --2- 172.21.15.45:0/541019899 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2570102420 0x7f2570102840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:22.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:39:22.159+0000 7f2574c2c700 1 --2- 172.21.15.45:0/541019899 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2570103620 0x7f2570103a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:22.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:39:22.159+0000 7f2574c2c700 1 -- 172.21.15.45:0/541019899 >> 172.21.15.45:0/541019899 conn(0x7f25700fd9c0 msgr2=0x7f25700ffe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:39:22.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:39:22.159+0000 7f2574c2c700 1 -- 172.21.15.45:0/541019899 shutdown_connections 2024-05-03T18:39:22.161 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:39:22.159+0000 7f2574c2c700 1 -- 172.21.15.45:0/541019899 wait complete. 2024-05-03T18:39:22.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:39:22.160+0000 7f2574c2c700 1 Processor -- start 2024-05-03T18:39:22.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:39:22.160+0000 7f2574c2c700 1 -- start start 2024-05-03T18:39:22.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:39:22.160+0000 7f2574c2c700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2570102420 0x7f25701995f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:39:22.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:39:22.160+0000 7f2574c2c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2570103620 0x7f2570199b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:39:22.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:39:22.160+0000 7f2574c2c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2570103fc0 0x7f25701a0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:39:22.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:39:22.160+0000 7f2574c2c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f257010deb0 con 0x7f2570103620 2024-05-03T18:39:22.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:39:22.160+0000 7f2574c2c700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f257010dd30 con 0x7f2570103fc0 2024-05-03T18:39:22.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:39:22.160+0000 7f2574c2c700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f257010e030 con 0x7f2570102420 2024-05-03T18:39:22.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:39:22.160+0000 7f256ed9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2570103fc0 0x7f25701a0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:39:22.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:39:22.160+0000 7f256ed9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2570103fc0 0x7f25701a0b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45374/0 (socket says 172.21.15.45:45374) 2024-05-03T18:39:22.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:39:22.160+0000 7f256ed9d700 1 -- 172.21.15.45:0/686749654 learned_addr learned my addr 172.21.15.45:0/686749654 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-03T18:39:22.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:39:22.160+0000 7f256e59c700 1 --2- 172.21.15.45:0/686749654 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2570102420 0x7f25701995f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:39:22.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:39:22.160+0000 7f256dd9b700 1 --2- 172.21.15.45:0/686749654 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2570103620 0x7f2570199b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:39:22.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:39:22.161+0000 7f256ed9d700 1 -- 172.21.15.45:0/686749654 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2570102420 msgr2=0x7f25701995f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:39:22.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:39:22.161+0000 7f256ed9d700 1 --2- 172.21.15.45:0/686749654 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2570102420 0x7f25701995f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:22.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:39:22.161+0000 7f256ed9d700 1 -- 172.21.15.45:0/686749654 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2570103620 msgr2=0x7f2570199b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:39:22.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:39:22.161+0000 7f256ed9d700 1 --2- 172.21.15.45:0/686749654 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2570103620 0x7f2570199b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:22.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:39:22.161+0000 7f256ed9d700 1 -- 172.21.15.45:0/686749654 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2570104840 con 0x7f2570103fc0 2024-05-03T18:39:22.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:39:22.161+0000 7f256ed9d700 1 --2- 172.21.15.45:0/686749654 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2570103fc0 0x7f25701a0b60 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f25680059d0 tx=0x7f25680047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:39:22.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:39:22.161+0000 7f25677fe700 1 -- 172.21.15.45:0/686749654 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2568003c30 con 0x7f2570103fc0 2024-05-03T18:39:22.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:39:22.161+0000 7f2574c2c700 1 -- 172.21.15.45:0/686749654 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2570104ad0 con 0x7f2570103fc0 2024-05-03T18:39:22.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:39:22.161+0000 7f25677fe700 1 -- 172.21.15.45:0/686749654 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2568004830 con 0x7f2570103fc0 2024-05-03T18:39:22.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:39:22.161+0000 7f25677fe700 1 -- 172.21.15.45:0/686749654 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2568005310 con 0x7f2570103fc0 2024-05-03T18:39:22.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:39:22.161+0000 7f2574c2c700 1 -- 172.21.15.45:0/686749654 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2570104fa0 con 0x7f2570103fc0 2024-05-03T18:39:22.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:39:22.162+0000 7f2574c2c700 1 -- 172.21.15.45:0/686749654 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f257006a190 con 0x7f2570103fc0 2024-05-03T18:39:22.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:39:22.163+0000 7f25677fe700 1 -- 172.21.15.45:0/686749654 <== mon.2 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 15) ==== 149596+0+0 (secure 0 0 0) 0x7f25680054b0 con 0x7f2570103fc0 2024-05-03T18:39:22.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:39:22.164+0000 7f25677fe700 1 --2- 172.21.15.45:0/686749654 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f255c0b2870 0x7f255c0b4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:39:22.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:39:22.164+0000 7f25677fe700 1 -- 172.21.15.45:0/686749654 <== mon.2 v2:172.21.15.45:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7f25680e1d10 con 0x7f2570103fc0 2024-05-03T18:39:22.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:39:22.164+0000 7f256e59c700 1 --2- 172.21.15.45:0/686749654 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f255c0b2870 0x7f255c0b4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:39:22.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:39:22.165+0000 7f256e59c700 1 --2- 172.21.15.45:0/686749654 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f255c0b2870 0x7f255c0b4d30 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f2560009550 tx=0x7f25600091b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:39:22.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:39:22.168+0000 7f25677fe700 1 -- 172.21.15.45:0/686749654 <== mon.2 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f2568013070 con 0x7f2570103fc0 2024-05-03T18:39:22.391 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:39:22.390+0000 7f2574c2c700 1 -- 172.21.15.45:0/686749654 --> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f25700ffbd0 con 0x7f255c0b2870 2024-05-03T18:39:23.313 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:23 smithi045 ceph-mon[33299]: pgmap v46: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-03T18:39:23.313 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:23 smithi045 ceph-mon[33299]: osdmap e9: 1 total, 1 up, 1 in 2024-05-03T18:39:23.313 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:23 smithi045 ceph-mon[33299]: from='client.24137 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:39:23.314 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:23 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T18:39:23.314 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:23 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T18:39:23.314 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:23 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:39:23.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:23 smithi113 ceph-mon[33540]: pgmap v46: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-03T18:39:23.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:23 smithi113 ceph-mon[33540]: osdmap e9: 1 total, 1 up, 1 in 2024-05-03T18:39:23.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:23 smithi113 ceph-mon[33540]: from='client.24137 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:39:23.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:23 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T18:39:23.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:23 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T18:39:23.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:23 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:39:23.567 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:23 smithi032 ceph-mon[29763]: pgmap v46: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-03T18:39:23.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:23 smithi032 ceph-mon[29763]: osdmap e9: 1 total, 1 up, 1 in 2024-05-03T18:39:23.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:23 smithi032 ceph-mon[29763]: from='client.24137 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:39:23.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:23 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T18:39:23.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:23 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T18:39:23.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:23 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:39:25.353 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:25 smithi113 ceph-mon[33540]: pgmap v48: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-03T18:39:25.353 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:25 smithi113 ceph-mon[33540]: from='client.? 172.21.15.45:0/1703855677' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ba8f0b67-1d7d-4d4c-8276-eff664d21f02"}]: dispatch 2024-05-03T18:39:25.353 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:25 smithi113 ceph-mon[33540]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ba8f0b67-1d7d-4d4c-8276-eff664d21f02"}]: dispatch 2024-05-03T18:39:25.353 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:25 smithi113 ceph-mon[33540]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ba8f0b67-1d7d-4d4c-8276-eff664d21f02"}]': finished 2024-05-03T18:39:25.354 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:25 smithi113 ceph-mon[33540]: osdmap e10: 2 total, 1 up, 2 in 2024-05-03T18:39:25.354 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:25 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-03T18:39:25.497 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:25 smithi045 ceph-mon[33299]: pgmap v48: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-03T18:39:25.497 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:25 smithi045 ceph-mon[33299]: from='client.? 172.21.15.45:0/1703855677' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ba8f0b67-1d7d-4d4c-8276-eff664d21f02"}]: dispatch 2024-05-03T18:39:25.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:25 smithi045 ceph-mon[33299]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ba8f0b67-1d7d-4d4c-8276-eff664d21f02"}]: dispatch 2024-05-03T18:39:25.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:25 smithi045 ceph-mon[33299]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ba8f0b67-1d7d-4d4c-8276-eff664d21f02"}]': finished 2024-05-03T18:39:25.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:25 smithi045 ceph-mon[33299]: osdmap e10: 2 total, 1 up, 2 in 2024-05-03T18:39:25.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:25 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-03T18:39:25.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:25 smithi032 ceph-mon[29763]: pgmap v48: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-03T18:39:25.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:25 smithi032 ceph-mon[29763]: from='client.? 172.21.15.45:0/1703855677' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ba8f0b67-1d7d-4d4c-8276-eff664d21f02"}]: dispatch 2024-05-03T18:39:25.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:25 smithi032 ceph-mon[29763]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ba8f0b67-1d7d-4d4c-8276-eff664d21f02"}]: dispatch 2024-05-03T18:39:25.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:25 smithi032 ceph-mon[29763]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ba8f0b67-1d7d-4d4c-8276-eff664d21f02"}]': finished 2024-05-03T18:39:25.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:25 smithi032 ceph-mon[29763]: osdmap e10: 2 total, 1 up, 2 in 2024-05-03T18:39:25.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:25 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-03T18:39:26.398 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:26 smithi045 ceph-mon[33299]: from='client.? 172.21.15.45:0/3913694733' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T18:39:26.534 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:26 smithi113 ceph-mon[33540]: from='client.? 172.21.15.45:0/3913694733' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T18:39:26.567 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:26 smithi032 ceph-mon[29763]: from='client.? 172.21.15.45:0/3913694733' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T18:39:27.497 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:27 smithi045 ceph-mon[33299]: pgmap v50: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-03T18:39:27.550 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:27 smithi113 ceph-mon[33540]: pgmap v50: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-03T18:39:27.567 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:27 smithi032 ceph-mon[29763]: pgmap v50: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-03T18:39:29.201 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:29 smithi045 ceph-mon[33299]: pgmap v51: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-03T18:39:29.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:29 smithi113 ceph-mon[33540]: pgmap v51: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-03T18:39:29.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:29 smithi032 ceph-mon[29763]: pgmap v51: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-03T18:39:31.250 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:31 smithi045 ceph-mon[33299]: pgmap v52: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-03T18:39:31.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:31 smithi113 ceph-mon[33540]: pgmap v52: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-03T18:39:31.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:31 smithi032 ceph-mon[29763]: pgmap v52: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-03T18:39:32.497 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:32 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-03T18:39:32.497 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:32 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:39:32.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:32 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-03T18:39:32.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:32 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:39:32.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:32 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-03T18:39:32.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:32 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:39:33.429 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:33 smithi045 ceph-mon[33299]: pgmap v53: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-03T18:39:33.429 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:33 smithi045 ceph-mon[33299]: Deploying daemon osd.1 on smithi045 2024-05-03T18:39:33.550 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:33 smithi113 ceph-mon[33540]: pgmap v53: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-03T18:39:33.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:33 smithi113 ceph-mon[33540]: Deploying daemon osd.1 on smithi045 2024-05-03T18:39:33.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:33 smithi032 ceph-mon[29763]: pgmap v53: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-03T18:39:33.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:33 smithi032 ceph-mon[29763]: Deploying daemon osd.1 on smithi045 2024-05-03T18:39:35.247 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:35 smithi045 ceph-mon[33299]: pgmap v54: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-03T18:39:35.370 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:35 smithi032 ceph-mon[29763]: pgmap v54: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-03T18:39:35.550 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:35 smithi113 ceph-mon[33540]: pgmap v54: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-03T18:39:37.497 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:37 smithi045 ceph-mon[33299]: pgmap v55: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-03T18:39:37.550 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:37 smithi113 ceph-mon[33540]: pgmap v55: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-03T18:39:37.569 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:37 smithi032 ceph-mon[29763]: pgmap v55: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-03T18:39:38.747 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:38 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:39:38.747 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:38 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:39:38.747 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:38 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:39:38.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:38 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:39:38.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:38 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:39:38.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:38 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:39:38.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:38 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:39:38.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:38 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:39:38.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:38 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:39:39.497 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:39 smithi045 ceph-mon[33299]: pgmap v56: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-03T18:39:39.800 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:39 smithi113 ceph-mon[33540]: pgmap v56: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-03T18:39:39.817 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:39 smithi032 ceph-mon[29763]: pgmap v56: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-03T18:39:40.713 INFO:teuthology.orchestra.run.smithi045.stdout:Created osd(s) 1 on host 'smithi045' 2024-05-03T18:39:40.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:39:40.712+0000 7f25677fe700 1 -- 172.21.15.45:0/686749654 <== mgr.14152 v2:172.21.15.32:6800/1796547904 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f25700ffbd0 con 0x7f255c0b2870 2024-05-03T18:39:40.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:39:40.714+0000 7f2574c2c700 1 -- 172.21.15.45:0/686749654 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f255c0b2870 msgr2=0x7f255c0b4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:39:40.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:39:40.714+0000 7f2574c2c700 1 --2- 172.21.15.45:0/686749654 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f255c0b2870 0x7f255c0b4d30 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f2560009550 tx=0x7f25600091b0 comp rx=0 tx=0).stop 2024-05-03T18:39:40.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:39:40.714+0000 7f2574c2c700 1 -- 172.21.15.45:0/686749654 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2570103fc0 msgr2=0x7f25701a0b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:39:40.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:39:40.714+0000 7f2574c2c700 1 --2- 172.21.15.45:0/686749654 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2570103fc0 0x7f25701a0b60 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f25680059d0 tx=0x7f25680047b0 comp rx=0 tx=0).stop 2024-05-03T18:39:40.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:39:40.715+0000 7f2574c2c700 1 -- 172.21.15.45:0/686749654 shutdown_connections 2024-05-03T18:39:40.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:39:40.715+0000 7f2574c2c700 1 --2- 172.21.15.45:0/686749654 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f255c0b2870 0x7f255c0b4d30 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:40.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:39:40.715+0000 7f2574c2c700 1 --2- 172.21.15.45:0/686749654 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2570103fc0 0x7f25701a0b60 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:40.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:39:40.715+0000 7f2574c2c700 1 --2- 172.21.15.45:0/686749654 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2570102420 0x7f25701995f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:40.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:39:40.715+0000 7f2574c2c700 1 --2- 172.21.15.45:0/686749654 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2570103620 0x7f2570199b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:40.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:39:40.715+0000 7f2574c2c700 1 -- 172.21.15.45:0/686749654 >> 172.21.15.45:0/686749654 conn(0x7f25700fd9c0 msgr2=0x7f25700ff4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:39:40.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:39:40.715+0000 7f2574c2c700 1 -- 172.21.15.45:0/686749654 shutdown_connections 2024-05-03T18:39:40.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:39:40.715+0000 7f2574c2c700 1 -- 172.21.15.45:0/686749654 wait complete. 2024-05-03T18:39:41.317 DEBUG:teuthology.orchestra.run.smithi045:osd.1> sudo journalctl -f -n 0 -u ceph-f500455e-097b-11ef-bc97-c7b262605968@osd.1.service 2024-05-03T18:39:41.319 INFO:tasks.cephadm:Deploying osd.2 on smithi113 with /dev/vg_nvme/lv_4... 2024-05-03T18:39:41.319 DEBUG:teuthology.orchestra.run.smithi113:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f500455e-097b-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-03T18:39:41.550 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:41 smithi113 ceph-mon[33540]: pgmap v57: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-03T18:39:41.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:41 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:39:41.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:41 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:39:41.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:41 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:39:41.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:41 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:39:41.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:41 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:39:41.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:41 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:39:41.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:41 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:39:41.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:41 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:39:41.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:41 smithi113 ceph-mon[33540]: from='osd.1 [v2:172.21.15.45:6800/1209907302,v1:172.21.15.45:6801/1209907302]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-03T18:39:41.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:41 smithi113 ceph-mon[33540]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-03T18:39:41.552 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:41 smithi045 ceph-mon[33299]: pgmap v57: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-03T18:39:41.552 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:41 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:39:41.552 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:41 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:39:41.552 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:41 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:39:41.552 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:41 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:39:41.552 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:41 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:39:41.552 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:41 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:39:41.552 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:41 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:39:41.552 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:41 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:39:41.553 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:41 smithi045 ceph-mon[33299]: from='osd.1 [v2:172.21.15.45:6800/1209907302,v1:172.21.15.45:6801/1209907302]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-03T18:39:41.553 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:41 smithi045 ceph-mon[33299]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-03T18:39:41.562 INFO:teuthology.orchestra.run.smithi113.stderr:Inferring config /var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/mon.c/config 2024-05-03T18:39:41.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:41 smithi032 ceph-mon[29763]: pgmap v57: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-03T18:39:41.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:41 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:39:41.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:41 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:39:41.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:41 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:39:41.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:41 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:39:41.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:41 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:39:41.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:41 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:39:41.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:41 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:39:41.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:41 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:39:41.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:41 smithi032 ceph-mon[29763]: from='osd.1 [v2:172.21.15.45:6800/1209907302,v1:172.21.15.45:6801/1209907302]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-03T18:39:41.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:41 smithi032 ceph-mon[29763]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-03T18:39:42.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:42 smithi045 ceph-mon[33299]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-03T18:39:42.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:42 smithi045 ceph-mon[33299]: from='osd.1 [v2:172.21.15.45:6800/1209907302,v1:172.21.15.45:6801/1209907302]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-05-03T18:39:42.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:42 smithi045 ceph-mon[33299]: osdmap e11: 2 total, 1 up, 2 in 2024-05-03T18:39:42.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:42 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-03T18:39:42.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:42 smithi045 ceph-mon[33299]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-05-03T18:39:42.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:42 smithi045 ceph-mon[33299]: pgmap v59: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-03T18:39:42.998 INFO:journalctl@ceph.osd.1.smithi045.stdout:May 03 18:39:42 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-1[37891]: 2024-05-03T18:39:42.693+0000 7f87e05bc700 -1 osd.1 0 waiting for initial osdmap 2024-05-03T18:39:42.998 INFO:journalctl@ceph.osd.1.smithi045.stdout:May 03 18:39:42 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-1[37891]: 2024-05-03T18:39:42.697+0000 7f87dc3af700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-03T18:39:43.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:42 smithi113 ceph-mon[33540]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-03T18:39:43.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:42 smithi113 ceph-mon[33540]: from='osd.1 [v2:172.21.15.45:6800/1209907302,v1:172.21.15.45:6801/1209907302]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-05-03T18:39:43.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:42 smithi113 ceph-mon[33540]: osdmap e11: 2 total, 1 up, 2 in 2024-05-03T18:39:43.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:42 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-03T18:39:43.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:42 smithi113 ceph-mon[33540]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-05-03T18:39:43.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:42 smithi113 ceph-mon[33540]: pgmap v59: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-03T18:39:43.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:42 smithi032 ceph-mon[29763]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-03T18:39:43.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:42 smithi032 ceph-mon[29763]: from='osd.1 [v2:172.21.15.45:6800/1209907302,v1:172.21.15.45:6801/1209907302]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-05-03T18:39:43.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:42 smithi032 ceph-mon[29763]: osdmap e11: 2 total, 1 up, 2 in 2024-05-03T18:39:43.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:42 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-03T18:39:43.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:42 smithi032 ceph-mon[29763]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-05-03T18:39:43.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:42 smithi032 ceph-mon[29763]: pgmap v59: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-03T18:39:43.262 INFO:teuthology.orchestra.run.smithi113.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-03T18:39:43.262 INFO:teuthology.orchestra.run.smithi113.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-03T18:39:43.263 INFO:teuthology.orchestra.run.smithi113.stderr: stderr: 10+0 records in 2024-05-03T18:39:43.263 INFO:teuthology.orchestra.run.smithi113.stderr:10+0 records out 2024-05-03T18:39:43.263 INFO:teuthology.orchestra.run.smithi113.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00853952 s, 1.2 GB/s 2024-05-03T18:39:43.263 INFO:teuthology.orchestra.run.smithi113.stderr:--> Zapping successful for: 2024-05-03T18:39:43.886 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:43 smithi113 ceph-mon[33540]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2024-05-03T18:39:43.886 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:43 smithi113 ceph-mon[33540]: osdmap e12: 2 total, 1 up, 2 in 2024-05-03T18:39:43.886 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:43 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-03T18:39:43.887 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:43 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-03T18:39:43.914 DEBUG:teuthology.orchestra.run.smithi113:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f500455e-097b-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi113:vg_nvme/lv_4 2024-05-03T18:39:43.932 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:43 smithi045 ceph-mon[33299]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2024-05-03T18:39:43.932 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:43 smithi045 ceph-mon[33299]: osdmap e12: 2 total, 1 up, 2 in 2024-05-03T18:39:43.932 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:43 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-03T18:39:43.932 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:43 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-03T18:39:44.067 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:43 smithi032 ceph-mon[29763]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2024-05-03T18:39:44.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:43 smithi032 ceph-mon[29763]: osdmap e12: 2 total, 1 up, 2 in 2024-05-03T18:39:44.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:43 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-03T18:39:44.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:43 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-03T18:39:44.156 INFO:teuthology.orchestra.run.smithi113.stderr:Inferring config /var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/mon.c/config 2024-05-03T18:39:44.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:44 smithi113 ceph-mon[33540]: purged_snaps scrub starts 2024-05-03T18:39:44.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:44 smithi113 ceph-mon[33540]: purged_snaps scrub ok 2024-05-03T18:39:44.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:44 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-03T18:39:44.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:44 smithi113 ceph-mon[33540]: osd.1 [v2:172.21.15.45:6800/1209907302,v1:172.21.15.45:6801/1209907302] boot 2024-05-03T18:39:44.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:44 smithi113 ceph-mon[33540]: osdmap e13: 2 total, 2 up, 2 in 2024-05-03T18:39:44.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:44 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-03T18:39:44.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:44 smithi113 ceph-mon[33540]: pgmap v62: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-03T18:39:44.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:44 smithi113 ceph-mon[33540]: Detected new or changed devices on smithi045 2024-05-03T18:39:44.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:44 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:39:44.802 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:44 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:39:44.802 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:44 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:39:44.802 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:44 smithi113 ceph-mon[33540]: Adjusting osd_memory_target on smithi045 to 17093M 2024-05-03T18:39:44.802 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:44 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:39:44.802 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:44 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:39:44.802 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:44 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:39:44.802 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:44 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:39:44.997 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:44 smithi045 ceph-mon[33299]: purged_snaps scrub starts 2024-05-03T18:39:44.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:44 smithi045 ceph-mon[33299]: purged_snaps scrub ok 2024-05-03T18:39:44.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:44 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-03T18:39:44.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:44 smithi045 ceph-mon[33299]: osd.1 [v2:172.21.15.45:6800/1209907302,v1:172.21.15.45:6801/1209907302] boot 2024-05-03T18:39:44.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:44 smithi045 ceph-mon[33299]: osdmap e13: 2 total, 2 up, 2 in 2024-05-03T18:39:44.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:44 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-03T18:39:44.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:44 smithi045 ceph-mon[33299]: pgmap v62: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-03T18:39:44.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:44 smithi045 ceph-mon[33299]: Detected new or changed devices on smithi045 2024-05-03T18:39:44.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:44 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:39:44.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:44 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:39:44.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:44 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:39:44.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:44 smithi045 ceph-mon[33299]: Adjusting osd_memory_target on smithi045 to 17093M 2024-05-03T18:39:44.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:44 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:39:44.999 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:44 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:39:44.999 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:44 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:39:44.999 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:44 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:39:45.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:44 smithi032 ceph-mon[29763]: purged_snaps scrub starts 2024-05-03T18:39:45.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:44 smithi032 ceph-mon[29763]: purged_snaps scrub ok 2024-05-03T18:39:45.069 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:44 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-03T18:39:45.069 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:44 smithi032 ceph-mon[29763]: osd.1 [v2:172.21.15.45:6800/1209907302,v1:172.21.15.45:6801/1209907302] boot 2024-05-03T18:39:45.069 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:44 smithi032 ceph-mon[29763]: osdmap e13: 2 total, 2 up, 2 in 2024-05-03T18:39:45.069 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:44 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-03T18:39:45.069 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:44 smithi032 ceph-mon[29763]: pgmap v62: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-05-03T18:39:45.069 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:44 smithi032 ceph-mon[29763]: Detected new or changed devices on smithi045 2024-05-03T18:39:45.069 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:44 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:39:45.069 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:44 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:39:45.069 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:44 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:39:45.069 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:44 smithi032 ceph-mon[29763]: Adjusting osd_memory_target on smithi045 to 17093M 2024-05-03T18:39:45.069 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:44 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:39:45.070 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:44 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:39:45.070 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:44 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:39:45.070 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:44 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:39:45.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:39:45.664+0000 7f6ad8f95700 1 -- 172.21.15.113:0/1890668192 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6ad4103040 msgr2=0x7f6ad410a900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:39:45.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:39:45.664+0000 7f6ad8f95700 1 --2- 172.21.15.113:0/1890668192 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6ad4103040 0x7f6ad410a900 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f6acc00b0a0 tx=0x7f6acc00b3b0 comp rx=0 tx=0).stop 2024-05-03T18:39:45.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:39:45.665+0000 7f6ad8f95700 1 -- 172.21.15.113:0/1890668192 shutdown_connections 2024-05-03T18:39:45.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:39:45.665+0000 7f6ad8f95700 1 --2- 172.21.15.113:0/1890668192 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6ad4103040 0x7f6ad410a900 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:45.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:39:45.665+0000 7f6ad8f95700 1 --2- 172.21.15.113:0/1890668192 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6ad41039f0 0x7f6ad4103e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:45.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:39:45.665+0000 7f6ad8f95700 1 --2- 172.21.15.113:0/1890668192 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ad41026a0 0x7f6ad4102b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:45.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:39:45.665+0000 7f6ad8f95700 1 -- 172.21.15.113:0/1890668192 >> 172.21.15.113:0/1890668192 conn(0x7f6ad40fd110 msgr2=0x7f6ad40ff570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:39:45.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:39:45.665+0000 7f6ad8f95700 1 -- 172.21.15.113:0/1890668192 shutdown_connections 2024-05-03T18:39:45.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:39:45.665+0000 7f6ad8f95700 1 -- 172.21.15.113:0/1890668192 wait complete. 2024-05-03T18:39:45.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:39:45.665+0000 7f6ad8f95700 1 Processor -- start 2024-05-03T18:39:45.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:39:45.665+0000 7f6ad8f95700 1 -- start start 2024-05-03T18:39:45.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:39:45.666+0000 7f6ad8f95700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6ad41026a0 0x7f6ad41996d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:39:45.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:39:45.666+0000 7f6ad8f95700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ad4103040 0x7f6ad4199c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:39:45.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:39:45.666+0000 7f6ad8f95700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6ad41039f0 0x7f6ad41a0c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:39:45.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:39:45.666+0000 7f6ad8f95700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f6ad410df70 con 0x7f6ad4103040 2024-05-03T18:39:45.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:39:45.666+0000 7f6ad8f95700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6ad410ddf0 con 0x7f6ad41026a0 2024-05-03T18:39:45.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:39:45.666+0000 7f6ad8f95700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f6ad410e0f0 con 0x7f6ad41039f0 2024-05-03T18:39:45.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:39:45.666+0000 7f6ad2d9d700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6ad41039f0 0x7f6ad41a0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:39:45.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:39:45.666+0000 7f6ad2d9d700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6ad41039f0 0x7f6ad41a0c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:52222/0 (socket says 172.21.15.113:52222) 2024-05-03T18:39:45.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:39:45.666+0000 7f6ad2d9d700 1 -- 172.21.15.113:0/2390675396 learned_addr learned my addr 172.21.15.113:0/2390675396 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-05-03T18:39:45.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:39:45.666+0000 7f6ad1d9b700 1 --2- 172.21.15.113:0/2390675396 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ad4103040 0x7f6ad4199c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:39:45.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:39:45.666+0000 7f6ad259c700 1 --2- 172.21.15.113:0/2390675396 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6ad41026a0 0x7f6ad41996d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:39:45.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:39:45.666+0000 7f6ad2d9d700 1 -- 172.21.15.113:0/2390675396 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6ad41026a0 msgr2=0x7f6ad41996d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:39:45.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:39:45.666+0000 7f6ad2d9d700 1 --2- 172.21.15.113:0/2390675396 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6ad41026a0 0x7f6ad41996d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:45.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:39:45.666+0000 7f6ad2d9d700 1 -- 172.21.15.113:0/2390675396 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ad4103040 msgr2=0x7f6ad4199c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:39:45.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:39:45.666+0000 7f6ad2d9d700 1 --2- 172.21.15.113:0/2390675396 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ad4103040 0x7f6ad4199c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:39:45.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:39:45.666+0000 7f6ad2d9d700 1 -- 172.21.15.113:0/2390675396 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ad41a1330 con 0x7f6ad41039f0 2024-05-03T18:39:45.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:39:45.666+0000 7f6ad259c700 1 --2- 172.21.15.113:0/2390675396 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6ad41026a0 0x7f6ad41996d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:39:45.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:39:45.666+0000 7f6ad2d9d700 1 --2- 172.21.15.113:0/2390675396 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6ad41039f0 0x7f6ad41a0c90 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f6acc00b070 tx=0x7f6acc004dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:39:45.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:39:45.666+0000 7f6ac37fe700 1 -- 172.21.15.113:0/2390675396 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6acc021070 con 0x7f6ad41039f0 2024-05-03T18:39:45.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:39:45.666+0000 7f6ac37fe700 1 -- 172.21.15.113:0/2390675396 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6acc007c80 con 0x7f6ad41039f0 2024-05-03T18:39:45.670 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:39:45.666+0000 7f6ac37fe700 1 -- 172.21.15.113:0/2390675396 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6acc005390 con 0x7f6ad41039f0 2024-05-03T18:39:45.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:39:45.666+0000 7f6ad8f95700 1 -- 172.21.15.113:0/2390675396 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ad41a15c0 con 0x7f6ad41039f0 2024-05-03T18:39:45.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:39:45.666+0000 7f6ad8f95700 1 -- 172.21.15.113:0/2390675396 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ad41a1b30 con 0x7f6ad41039f0 2024-05-03T18:39:45.671 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:39:45.668+0000 7f6ac17fa700 1 -- 172.21.15.113:0/2390675396 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ab8005330 con 0x7f6ad41039f0 2024-05-03T18:39:45.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:39:45.669+0000 7f6ac37fe700 1 -- 172.21.15.113:0/2390675396 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 15) ==== 149596+0+0 (secure 0 0 0) 0x7f6acc024070 con 0x7f6ad41039f0 2024-05-03T18:39:45.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:39:45.669+0000 7f6ac37fe700 1 --2- 172.21.15.113:0/2390675396 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f6abc0b2870 0x7f6abc0b4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:39:45.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:39:45.669+0000 7f6ac37fe700 1 -- 172.21.15.113:0/2390675396 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2309+0+0 (secure 0 0 0) 0x7f6acc0e1750 con 0x7f6ad41039f0 2024-05-03T18:39:45.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:39:45.669+0000 7f6ad259c700 1 --2- 172.21.15.113:0/2390675396 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f6abc0b2870 0x7f6abc0b4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:39:45.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:39:45.670+0000 7f6ad259c700 1 --2- 172.21.15.113:0/2390675396 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f6abc0b2870 0x7f6abc0b4d30 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6ad406ba10 tx=0x7f6ac4008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:39:45.674 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:39:45.674+0000 7f6ac37fe700 1 -- 172.21.15.113:0/2390675396 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f6acc0aad50 con 0x7f6ad41039f0 2024-05-03T18:39:45.892 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:39:45.891+0000 7f6ac17fa700 1 -- 172.21.15.113:0/2390675396 --> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi113:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f6ab8000bc0 con 0x7f6abc0b2870 2024-05-03T18:39:46.300 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:46 smithi113 ceph-mon[33540]: osdmap e14: 2 total, 2 up, 2 in 2024-05-03T18:39:46.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:46 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T18:39:46.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:46 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T18:39:46.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:46 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:39:46.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:46 smithi032 ceph-mon[29763]: osdmap e14: 2 total, 2 up, 2 in 2024-05-03T18:39:46.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:46 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T18:39:46.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:46 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T18:39:46.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:46 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:39:46.497 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:46 smithi045 ceph-mon[33299]: osdmap e14: 2 total, 2 up, 2 in 2024-05-03T18:39:46.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:46 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T18:39:46.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:46 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T18:39:46.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:46 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:39:47.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:47 smithi113 ceph-mon[33540]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi113:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:39:47.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:47 smithi113 ceph-mon[33540]: pgmap v64: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-03T18:39:47.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:47 smithi032 ceph-mon[29763]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi113:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:39:47.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:47 smithi032 ceph-mon[29763]: pgmap v64: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-03T18:39:47.497 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:47 smithi045 ceph-mon[33299]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi113:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:39:47.497 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:47 smithi045 ceph-mon[33299]: pgmap v64: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-03T18:39:49.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:49 smithi113 ceph-mon[33540]: pgmap v65: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-03T18:39:49.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:49 smithi113 ceph-mon[33540]: from='client.? 172.21.15.113:0/2615595659' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5624eabc-be47-4bb8-9d8e-b84734769c83"}]: dispatch 2024-05-03T18:39:49.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:49 smithi113 ceph-mon[33540]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5624eabc-be47-4bb8-9d8e-b84734769c83"}]: dispatch 2024-05-03T18:39:49.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:49 smithi113 ceph-mon[33540]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5624eabc-be47-4bb8-9d8e-b84734769c83"}]': finished 2024-05-03T18:39:49.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:49 smithi113 ceph-mon[33540]: osdmap e15: 3 total, 2 up, 3 in 2024-05-03T18:39:49.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:49 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-03T18:39:49.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:49 smithi032 ceph-mon[29763]: pgmap v65: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-03T18:39:49.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:49 smithi032 ceph-mon[29763]: from='client.? 172.21.15.113:0/2615595659' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5624eabc-be47-4bb8-9d8e-b84734769c83"}]: dispatch 2024-05-03T18:39:49.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:49 smithi032 ceph-mon[29763]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5624eabc-be47-4bb8-9d8e-b84734769c83"}]: dispatch 2024-05-03T18:39:49.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:49 smithi032 ceph-mon[29763]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5624eabc-be47-4bb8-9d8e-b84734769c83"}]': finished 2024-05-03T18:39:49.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:49 smithi032 ceph-mon[29763]: osdmap e15: 3 total, 2 up, 3 in 2024-05-03T18:39:49.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:49 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-03T18:39:49.497 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:49 smithi045 ceph-mon[33299]: pgmap v65: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-03T18:39:49.497 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:49 smithi045 ceph-mon[33299]: from='client.? 172.21.15.113:0/2615595659' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5624eabc-be47-4bb8-9d8e-b84734769c83"}]: dispatch 2024-05-03T18:39:49.497 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:49 smithi045 ceph-mon[33299]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5624eabc-be47-4bb8-9d8e-b84734769c83"}]: dispatch 2024-05-03T18:39:49.497 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:49 smithi045 ceph-mon[33299]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5624eabc-be47-4bb8-9d8e-b84734769c83"}]': finished 2024-05-03T18:39:49.497 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:49 smithi045 ceph-mon[33299]: osdmap e15: 3 total, 2 up, 3 in 2024-05-03T18:39:49.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:49 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-03T18:39:50.300 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:50 smithi113 ceph-mon[33540]: from='client.? 172.21.15.113:0/977226245' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T18:39:50.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:50 smithi032 ceph-mon[29763]: from='client.? 172.21.15.113:0/977226245' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T18:39:50.497 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:50 smithi045 ceph-mon[33299]: from='client.? 172.21.15.113:0/977226245' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T18:39:51.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:51 smithi113 ceph-mon[33540]: pgmap v67: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-03T18:39:51.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:51 smithi032 ceph-mon[29763]: pgmap v67: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-03T18:39:51.497 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:51 smithi045 ceph-mon[33299]: pgmap v67: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-03T18:39:53.300 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:53 smithi113 ceph-mon[33540]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-03T18:39:53.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:53 smithi032 ceph-mon[29763]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-03T18:39:53.497 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:53 smithi045 ceph-mon[33299]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-03T18:39:55.270 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:55 smithi113 ceph-mon[33540]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-03T18:39:55.317 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:55 smithi032 ceph-mon[29763]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-03T18:39:55.497 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:55 smithi045 ceph-mon[33299]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-03T18:39:56.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:56 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-03T18:39:56.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:56 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:39:56.317 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:56 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-03T18:39:56.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:56 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:39:56.398 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:56 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-03T18:39:56.399 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:56 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:39:57.256 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:57 smithi113 ceph-mon[33540]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-03T18:39:57.256 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:57 smithi113 ceph-mon[33540]: Deploying daemon osd.2 on smithi113 2024-05-03T18:39:57.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:57 smithi032 ceph-mon[29763]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-03T18:39:57.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:57 smithi032 ceph-mon[29763]: Deploying daemon osd.2 on smithi113 2024-05-03T18:39:57.497 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:57 smithi045 ceph-mon[33299]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-03T18:39:57.497 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:57 smithi045 ceph-mon[33299]: Deploying daemon osd.2 on smithi113 2024-05-03T18:39:59.317 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:39:59 smithi113 ceph-mon[33540]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-03T18:39:59.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:39:59 smithi032 ceph-mon[29763]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-03T18:39:59.497 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:39:59 smithi045 ceph-mon[33299]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-03T18:40:00.171 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:00 smithi113 ceph-mon[33540]: overall HEALTH_OK 2024-05-03T18:40:00.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:00 smithi032 ceph-mon[29763]: overall HEALTH_OK 2024-05-03T18:40:00.497 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:00 smithi045 ceph-mon[33299]: overall HEALTH_OK 2024-05-03T18:40:01.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:01 smithi113 ceph-mon[33540]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-03T18:40:01.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:01 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:40:01.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:01 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:40:01.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:01 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:40:01.317 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:01 smithi032 ceph-mon[29763]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-03T18:40:01.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:01 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:40:01.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:01 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:40:01.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:01 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:40:01.497 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:01 smithi045 ceph-mon[33299]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-03T18:40:01.497 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:01 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:40:01.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:01 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:40:01.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:01 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:40:03.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:03 smithi113 ceph-mon[33540]: pgmap v73: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-03T18:40:03.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:03 smithi032 ceph-mon[29763]: pgmap v73: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-03T18:40:03.497 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:03 smithi045 ceph-mon[33299]: pgmap v73: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-03T18:40:04.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:04 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:40:04.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:04 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:40:04.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:04 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:40:04.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:04 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:40:04.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:04 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:40:04.550 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:04 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:40:04.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:04 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:40:04.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:04 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:40:04.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:04 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:40:04.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:04 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:40:04.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:04 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:40:04.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:04 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:40:04.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:04 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:40:04.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:04 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:40:04.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:04 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:40:04.612 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:40:04.611+0000 7f6ac37fe700 1 -- 172.21.15.113:0/2390675396 <== mgr.14152 v2:172.21.15.32:6800/1796547904 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f6ab8000bc0 con 0x7f6abc0b2870 2024-05-03T18:40:04.612 INFO:teuthology.orchestra.run.smithi113.stdout:Created osd(s) 2 on host 'smithi113' 2024-05-03T18:40:04.614 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:40:04.614+0000 7f6ac17fa700 1 -- 172.21.15.113:0/2390675396 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f6abc0b2870 msgr2=0x7f6abc0b4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:40:04.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:40:04.614+0000 7f6ac17fa700 1 --2- 172.21.15.113:0/2390675396 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f6abc0b2870 0x7f6abc0b4d30 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6ad406ba10 tx=0x7f6ac4008040 comp rx=0 tx=0).stop 2024-05-03T18:40:04.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:40:04.614+0000 7f6ac17fa700 1 -- 172.21.15.113:0/2390675396 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6ad41039f0 msgr2=0x7f6ad41a0c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:40:04.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:40:04.614+0000 7f6ac17fa700 1 --2- 172.21.15.113:0/2390675396 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6ad41039f0 0x7f6ad41a0c90 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f6acc00b070 tx=0x7f6acc004dc0 comp rx=0 tx=0).stop 2024-05-03T18:40:04.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:40:04.614+0000 7f6ac17fa700 1 -- 172.21.15.113:0/2390675396 shutdown_connections 2024-05-03T18:40:04.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:40:04.614+0000 7f6ac17fa700 1 --2- 172.21.15.113:0/2390675396 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f6abc0b2870 0x7f6abc0b4d30 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:04.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:40:04.614+0000 7f6ac17fa700 1 --2- 172.21.15.113:0/2390675396 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6ad41039f0 0x7f6ad41a0c90 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:04.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:40:04.614+0000 7f6ac17fa700 1 --2- 172.21.15.113:0/2390675396 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6ad41026a0 0x7f6ad41996d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:04.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:40:04.614+0000 7f6ac17fa700 1 --2- 172.21.15.113:0/2390675396 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ad4103040 0x7f6ad4199c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:04.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:40:04.614+0000 7f6ac17fa700 1 -- 172.21.15.113:0/2390675396 >> 172.21.15.113:0/2390675396 conn(0x7f6ad40fd110 msgr2=0x7f6ad40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:40:04.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:40:04.614+0000 7f6ac17fa700 1 -- 172.21.15.113:0/2390675396 shutdown_connections 2024-05-03T18:40:04.615 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:40:04.614+0000 7f6ac17fa700 1 -- 172.21.15.113:0/2390675396 wait complete. 2024-05-03T18:40:05.160 DEBUG:teuthology.orchestra.run.smithi113:osd.2> sudo journalctl -f -n 0 -u ceph-f500455e-097b-11ef-bc97-c7b262605968@osd.2.service 2024-05-03T18:40:05.163 INFO:tasks.cephadm:Deploying osd.3 on smithi032 with /dev/vg_nvme/lv_3... 2024-05-03T18:40:05.163 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f500455e-097b-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-03T18:40:05.420 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:40:05.447 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:05 smithi113 ceph-mon[33540]: pgmap v74: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-03T18:40:05.447 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:05 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:40:05.447 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:05 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:40:05.447 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:05 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:40:05.447 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:05 smithi113 ceph-mon[33540]: from='osd.2 [v2:172.21.15.113:6800/2308128926,v1:172.21.15.113:6801/2308128926]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-03T18:40:05.447 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:05 smithi113 ceph-mon[33540]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-03T18:40:05.469 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:05 smithi032 ceph-mon[29763]: pgmap v74: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-03T18:40:05.469 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:05 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:40:05.470 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:05 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:40:05.470 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:05 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:40:05.470 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:05 smithi032 ceph-mon[29763]: from='osd.2 [v2:172.21.15.113:6800/2308128926,v1:172.21.15.113:6801/2308128926]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-03T18:40:05.470 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:05 smithi032 ceph-mon[29763]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-03T18:40:05.747 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:05 smithi045 ceph-mon[33299]: pgmap v74: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-03T18:40:05.747 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:05 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:40:05.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:05 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:40:05.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:05 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:40:05.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:05 smithi045 ceph-mon[33299]: from='osd.2 [v2:172.21.15.113:6800/2308128926,v1:172.21.15.113:6801/2308128926]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-03T18:40:05.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:05 smithi045 ceph-mon[33299]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-03T18:40:06.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:06 smithi113 ceph-mon[33540]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-03T18:40:06.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:06 smithi113 ceph-mon[33540]: osdmap e16: 3 total, 2 up, 3 in 2024-05-03T18:40:06.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:06 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-03T18:40:06.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:06 smithi113 ceph-mon[33540]: from='osd.2 [v2:172.21.15.113:6800/2308128926,v1:172.21.15.113:6801/2308128926]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi113", "root=default"]}]: dispatch 2024-05-03T18:40:06.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:06 smithi113 ceph-mon[33540]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi113", "root=default"]}]: dispatch 2024-05-03T18:40:06.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:06 smithi113 ceph-mon[33540]: pgmap v76: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-03T18:40:06.801 INFO:journalctl@ceph.osd.2.smithi113.stdout:May 03 18:40:06 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-2[38708]: 2024-05-03T18:40:06.598+0000 7faf354c7700 -1 osd.2 0 waiting for initial osdmap 2024-05-03T18:40:06.802 INFO:journalctl@ceph.osd.2.smithi113.stdout:May 03 18:40:06 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-2[38708]: 2024-05-03T18:40:06.602+0000 7faf312ba700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-03T18:40:06.854 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:06 smithi032 ceph-mon[29763]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-03T18:40:06.854 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:06 smithi032 ceph-mon[29763]: osdmap e16: 3 total, 2 up, 3 in 2024-05-03T18:40:06.855 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:06 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-03T18:40:06.855 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:06 smithi032 ceph-mon[29763]: from='osd.2 [v2:172.21.15.113:6800/2308128926,v1:172.21.15.113:6801/2308128926]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi113", "root=default"]}]: dispatch 2024-05-03T18:40:06.855 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:06 smithi032 ceph-mon[29763]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi113", "root=default"]}]: dispatch 2024-05-03T18:40:06.855 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:06 smithi032 ceph-mon[29763]: pgmap v76: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-03T18:40:06.997 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:06 smithi045 ceph-mon[33299]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-03T18:40:06.997 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:06 smithi045 ceph-mon[33299]: osdmap e16: 3 total, 2 up, 3 in 2024-05-03T18:40:06.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:06 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-03T18:40:06.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:06 smithi045 ceph-mon[33299]: from='osd.2 [v2:172.21.15.113:6800/2308128926,v1:172.21.15.113:6801/2308128926]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi113", "root=default"]}]: dispatch 2024-05-03T18:40:06.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:06 smithi045 ceph-mon[33299]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi113", "root=default"]}]: dispatch 2024-05-03T18:40:06.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:06 smithi045 ceph-mon[33299]: pgmap v76: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-03T18:40:07.542 INFO:teuthology.orchestra.run.smithi032.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-03T18:40:07.542 INFO:teuthology.orchestra.run.smithi032.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-03T18:40:07.542 INFO:teuthology.orchestra.run.smithi032.stderr: stderr: 10+0 records in 2024-05-03T18:40:07.542 INFO:teuthology.orchestra.run.smithi032.stderr:10+0 records out 2024-05-03T18:40:07.543 INFO:teuthology.orchestra.run.smithi032.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0081912 s, 1.3 GB/s 2024-05-03T18:40:07.543 INFO:teuthology.orchestra.run.smithi032.stderr:--> Zapping successful for: 2024-05-03T18:40:07.785 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:07 smithi113 ceph-mon[33540]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi113", "root=default"]}]': finished 2024-05-03T18:40:07.785 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:07 smithi113 ceph-mon[33540]: osdmap e17: 3 total, 2 up, 3 in 2024-05-03T18:40:07.785 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:07 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-03T18:40:07.785 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:07 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-03T18:40:07.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:07 smithi032 ceph-mon[29763]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi113", "root=default"]}]': finished 2024-05-03T18:40:07.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:07 smithi032 ceph-mon[29763]: osdmap e17: 3 total, 2 up, 3 in 2024-05-03T18:40:07.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:07 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-03T18:40:07.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:07 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-03T18:40:07.997 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:07 smithi045 ceph-mon[33299]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi113", "root=default"]}]': finished 2024-05-03T18:40:07.997 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:07 smithi045 ceph-mon[33299]: osdmap e17: 3 total, 2 up, 3 in 2024-05-03T18:40:07.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:07 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-03T18:40:07.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:07 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-03T18:40:08.198 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f500455e-097b-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi032:vg_nvme/lv_3 2024-05-03T18:40:08.462 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:40:08.862 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:08 smithi032 ceph-mon[29763]: purged_snaps scrub starts 2024-05-03T18:40:08.862 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:08 smithi032 ceph-mon[29763]: purged_snaps scrub ok 2024-05-03T18:40:08.862 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:08 smithi032 ceph-mon[29763]: osd.2 [v2:172.21.15.113:6800/2308128926,v1:172.21.15.113:6801/2308128926] boot 2024-05-03T18:40:08.862 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:08 smithi032 ceph-mon[29763]: osdmap e18: 3 total, 3 up, 3 in 2024-05-03T18:40:08.862 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:08 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-03T18:40:08.863 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:08 smithi032 ceph-mon[29763]: Detected new or changed devices on smithi113 2024-05-03T18:40:08.863 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:08 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:40:08.863 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:08 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:40:08.863 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:08 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:40:08.863 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:08 smithi032 ceph-mon[29763]: Adjusting osd_memory_target on smithi113 to 17088M 2024-05-03T18:40:08.863 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:08 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:40:08.863 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:08 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:40:08.863 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:08 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:40:08.863 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:08 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:40:08.863 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:08 smithi032 ceph-mon[29763]: pgmap v79: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-03T18:40:08.863 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:08 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-03T18:40:08.997 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:08 smithi045 ceph-mon[33299]: purged_snaps scrub starts 2024-05-03T18:40:09.032 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:08 smithi045 ceph-mon[33299]: purged_snaps scrub ok 2024-05-03T18:40:09.032 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:08 smithi045 ceph-mon[33299]: osd.2 [v2:172.21.15.113:6800/2308128926,v1:172.21.15.113:6801/2308128926] boot 2024-05-03T18:40:09.032 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:08 smithi045 ceph-mon[33299]: osdmap e18: 3 total, 3 up, 3 in 2024-05-03T18:40:09.032 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:08 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-03T18:40:09.032 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:08 smithi045 ceph-mon[33299]: Detected new or changed devices on smithi113 2024-05-03T18:40:09.032 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:08 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:40:09.032 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:08 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:40:09.032 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:08 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:40:09.032 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:08 smithi045 ceph-mon[33299]: Adjusting osd_memory_target on smithi113 to 17088M 2024-05-03T18:40:09.032 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:08 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:40:09.032 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:08 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:40:09.032 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:08 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:40:09.032 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:08 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:40:09.033 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:08 smithi045 ceph-mon[33299]: pgmap v79: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-03T18:40:09.033 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:08 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-03T18:40:09.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:08 smithi113 ceph-mon[33540]: purged_snaps scrub starts 2024-05-03T18:40:09.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:08 smithi113 ceph-mon[33540]: purged_snaps scrub ok 2024-05-03T18:40:09.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:08 smithi113 ceph-mon[33540]: osd.2 [v2:172.21.15.113:6800/2308128926,v1:172.21.15.113:6801/2308128926] boot 2024-05-03T18:40:09.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:08 smithi113 ceph-mon[33540]: osdmap e18: 3 total, 3 up, 3 in 2024-05-03T18:40:09.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:08 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-03T18:40:09.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:08 smithi113 ceph-mon[33540]: Detected new or changed devices on smithi113 2024-05-03T18:40:09.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:08 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:40:09.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:08 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:40:09.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:08 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:40:09.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:08 smithi113 ceph-mon[33540]: Adjusting osd_memory_target on smithi113 to 17088M 2024-05-03T18:40:09.052 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:08 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:40:09.052 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:08 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:40:09.052 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:08 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:40:09.052 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:08 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:40:09.052 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:08 smithi113 ceph-mon[33540]: pgmap v79: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-05-03T18:40:09.052 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:08 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-03T18:40:09.780 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:09 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-03T18:40:09.780 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:09 smithi032 ceph-mon[29763]: osdmap e19: 3 total, 3 up, 3 in 2024-05-03T18:40:09.780 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:09 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-03T18:40:09.780 INFO:journalctl@ceph.osd.0.smithi032.stdout:May 03 18:40:09 smithi032 sudo[46918]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-03T18:40:09.780 INFO:journalctl@ceph.osd.0.smithi032.stdout:May 03 18:40:09 smithi032 sudo[46918]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-03T18:40:09.780 INFO:journalctl@ceph.osd.0.smithi032.stdout:May 03 18:40:09 smithi032 sudo[46918]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-03T18:40:09.780 INFO:journalctl@ceph.osd.0.smithi032.stdout:May 03 18:40:09 smithi032 sudo[46918]: pam_unix(sudo:session): session closed for user root 2024-05-03T18:40:09.781 INFO:journalctl@ceph.osd.0.smithi032.stdout:May 03 18:40:09 smithi032 sudo[46921]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-03T18:40:09.781 INFO:journalctl@ceph.osd.0.smithi032.stdout:May 03 18:40:09 smithi032 sudo[46921]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-03T18:40:09.781 INFO:journalctl@ceph.osd.0.smithi032.stdout:May 03 18:40:09 smithi032 sudo[46921]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-03T18:40:09.781 INFO:journalctl@ceph.osd.0.smithi032.stdout:May 03 18:40:09 smithi032 sudo[46921]: pam_unix(sudo:session): session closed for user root 2024-05-03T18:40:09.900 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:09 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-03T18:40:09.900 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:09 smithi113 ceph-mon[33540]: osdmap e19: 3 total, 3 up, 3 in 2024-05-03T18:40:09.900 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:09 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-03T18:40:09.900 INFO:journalctl@ceph.osd.2.smithi113.stdout:May 03 18:40:09 smithi113 sudo[42767]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-03T18:40:09.900 INFO:journalctl@ceph.osd.2.smithi113.stdout:May 03 18:40:09 smithi113 sudo[42767]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-03T18:40:09.900 INFO:journalctl@ceph.osd.2.smithi113.stdout:May 03 18:40:09 smithi113 sudo[42767]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-03T18:40:09.900 INFO:journalctl@ceph.osd.2.smithi113.stdout:May 03 18:40:09 smithi113 sudo[42767]: pam_unix(sudo:session): session closed for user root 2024-05-03T18:40:09.900 INFO:journalctl@ceph.osd.2.smithi113.stdout:May 03 18:40:09 smithi113 sudo[42770]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-03T18:40:09.900 INFO:journalctl@ceph.osd.2.smithi113.stdout:May 03 18:40:09 smithi113 sudo[42770]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-03T18:40:09.900 INFO:journalctl@ceph.osd.2.smithi113.stdout:May 03 18:40:09 smithi113 sudo[42770]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-03T18:40:09.900 INFO:journalctl@ceph.osd.2.smithi113.stdout:May 03 18:40:09 smithi113 sudo[42770]: pam_unix(sudo:session): session closed for user root 2024-05-03T18:40:09.938 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:09 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-03T18:40:09.938 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:09 smithi045 ceph-mon[33299]: osdmap e19: 3 total, 3 up, 3 in 2024-05-03T18:40:09.938 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:09 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-03T18:40:09.938 INFO:journalctl@ceph.osd.1.smithi045.stdout:May 03 18:40:09 smithi045 sudo[42026]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-03T18:40:09.938 INFO:journalctl@ceph.osd.1.smithi045.stdout:May 03 18:40:09 smithi045 sudo[42026]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-03T18:40:09.938 INFO:journalctl@ceph.osd.1.smithi045.stdout:May 03 18:40:09 smithi045 sudo[42026]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-03T18:40:09.939 INFO:journalctl@ceph.osd.1.smithi045.stdout:May 03 18:40:09 smithi045 sudo[42026]: pam_unix(sudo:session): session closed for user root 2024-05-03T18:40:09.939 INFO:journalctl@ceph.osd.1.smithi045.stdout:May 03 18:40:09 smithi045 sudo[42029]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-03T18:40:09.939 INFO:journalctl@ceph.osd.1.smithi045.stdout:May 03 18:40:09 smithi045 sudo[42029]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-03T18:40:09.939 INFO:journalctl@ceph.osd.1.smithi045.stdout:May 03 18:40:09 smithi045 sudo[42029]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-03T18:40:09.939 INFO:journalctl@ceph.osd.1.smithi045.stdout:May 03 18:40:09 smithi045 sudo[42029]: pam_unix(sudo:session): session closed for user root 2024-05-03T18:40:09.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:40:09.992+0000 7f51d6c5e700 1 -- 172.21.15.32:0/988517308 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51d0104740 msgr2=0x7f51d0104ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:40:09.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:40:09.992+0000 7f51d6c5e700 1 --2- 172.21.15.32:0/988517308 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51d0104740 0x7f51d0104ba0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f51c40099c0 tx=0x7f51c4009cd0 comp rx=0 tx=0).stop 2024-05-03T18:40:09.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:40:09.992+0000 7f51d6c5e700 1 -- 172.21.15.32:0/988517308 shutdown_connections 2024-05-03T18:40:09.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:40:09.992+0000 7f51d6c5e700 1 --2- 172.21.15.32:0/988517308 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f51d01050e0 0x7f51d010b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:09.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:40:09.992+0000 7f51d6c5e700 1 --2- 172.21.15.32:0/988517308 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f51d0103540 0x7f51d0103960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:09.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:40:09.992+0000 7f51d6c5e700 1 --2- 172.21.15.32:0/988517308 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51d0104740 0x7f51d0104ba0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:09.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:40:09.992+0000 7f51d6c5e700 1 -- 172.21.15.32:0/988517308 >> 172.21.15.32:0/988517308 conn(0x7f51d00feae0 msgr2=0x7f51d0100f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:40:09.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:40:09.992+0000 7f51d6c5e700 1 -- 172.21.15.32:0/988517308 shutdown_connections 2024-05-03T18:40:09.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:40:09.992+0000 7f51d6c5e700 1 -- 172.21.15.32:0/988517308 wait complete. 2024-05-03T18:40:09.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:40:09.993+0000 7f51d6c5e700 1 Processor -- start 2024-05-03T18:40:09.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:40:09.993+0000 7f51d6c5e700 1 -- start start 2024-05-03T18:40:09.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:40:09.993+0000 7f51d6c5e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51d0103540 0x7f51d0077240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:40:09.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:40:09.993+0000 7f51d6c5e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f51d0104740 0x7f51d0075890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:40:09.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:40:09.993+0000 7f51d6c5e700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f51d01050e0 0x7f51d0075dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:40:09.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:40:09.993+0000 7f51d6c5e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f51d010efe0 con 0x7f51d0103540 2024-05-03T18:40:09.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:40:09.993+0000 7f51d6c5e700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f51d010ee60 con 0x7f51d0104740 2024-05-03T18:40:09.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:40:09.993+0000 7f51d6c5e700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f51d010f160 con 0x7f51d01050e0 2024-05-03T18:40:09.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:40:09.993+0000 7f51d49fa700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51d0103540 0x7f51d0077240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:40:09.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:40:09.993+0000 7f51d49fa700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51d0103540 0x7f51d0077240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43694/0 (socket says 172.21.15.32:43694) 2024-05-03T18:40:09.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:40:09.993+0000 7f51d49fa700 1 -- 172.21.15.32:0/259138957 learned_addr learned my addr 172.21.15.32:0/259138957 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:40:09.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:40:09.993+0000 7f51d51fb700 1 --2- 172.21.15.32:0/259138957 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f51d01050e0 0x7f51d0075dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:40:09.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:40:09.994+0000 7f51cffff700 1 --2- 172.21.15.32:0/259138957 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f51d0104740 0x7f51d0075890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:40:09.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:40:09.994+0000 7f51d49fa700 1 -- 172.21.15.32:0/259138957 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f51d01050e0 msgr2=0x7f51d0075dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:40:09.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:40:09.994+0000 7f51d49fa700 1 --2- 172.21.15.32:0/259138957 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f51d01050e0 0x7f51d0075dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:09.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:40:09.994+0000 7f51d49fa700 1 -- 172.21.15.32:0/259138957 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f51d0104740 msgr2=0x7f51d0075890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:40:09.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:40:09.994+0000 7f51d49fa700 1 --2- 172.21.15.32:0/259138957 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f51d0104740 0x7f51d0075890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:09.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:40:09.994+0000 7f51d49fa700 1 -- 172.21.15.32:0/259138957 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51d0076680 con 0x7f51d0103540 2024-05-03T18:40:09.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:40:09.994+0000 7f51d49fa700 1 --2- 172.21.15.32:0/259138957 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51d0103540 0x7f51d0077240 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f51c000c970 tx=0x7f51c000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:40:09.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:40:09.994+0000 7f51cdffb700 1 -- 172.21.15.32:0/259138957 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51c0004220 con 0x7f51d0103540 2024-05-03T18:40:09.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:40:09.994+0000 7f51cdffb700 1 -- 172.21.15.32:0/259138957 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f51c0007d50 con 0x7f51d0103540 2024-05-03T18:40:09.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:40:09.994+0000 7f51d6c5e700 1 -- 172.21.15.32:0/259138957 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51d01a37b0 con 0x7f51d0103540 2024-05-03T18:40:09.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:40:09.994+0000 7f51cdffb700 1 -- 172.21.15.32:0/259138957 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51c0003d00 con 0x7f51d0103540 2024-05-03T18:40:09.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:40:09.994+0000 7f51d6c5e700 1 -- 172.21.15.32:0/259138957 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51d01a3cf0 con 0x7f51d0103540 2024-05-03T18:40:09.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:40:09.995+0000 7f51d6c5e700 1 -- 172.21.15.32:0/259138957 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51d006a190 con 0x7f51d0103540 2024-05-03T18:40:09.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:40:09.997+0000 7f51cdffb700 1 -- 172.21.15.32:0/259138957 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 15) ==== 149596+0+0 (secure 0 0 0) 0x7f51c00078a0 con 0x7f51d0103540 2024-05-03T18:40:09.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:40:09.997+0000 7f51cdffb700 1 --2- 172.21.15.32:0/259138957 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f51b80b2870 0x7f51b80b4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:40:09.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:40:09.997+0000 7f51cffff700 1 --2- 172.21.15.32:0/259138957 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f51b80b2870 0x7f51b80b4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:40:09.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:40:09.997+0000 7f51cdffb700 1 -- 172.21.15.32:0/259138957 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 3237+0+0 (secure 0 0 0) 0x7f51c00e0c40 con 0x7f51d0103540 2024-05-03T18:40:09.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:40:09.998+0000 7f51cffff700 1 --2- 172.21.15.32:0/259138957 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f51b80b2870 0x7f51b80b4d30 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f51c4000c00 tx=0x7f51c4005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:40:10.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:40:10.001+0000 7f51cdffb700 1 -- 172.21.15.32:0/259138957 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f51c0014030 con 0x7f51d0103540 2024-05-03T18:40:10.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:09 smithi032 sudo[46929]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-03T18:40:10.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:09 smithi032 sudo[46929]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-03T18:40:10.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:09 smithi032 sudo[46929]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-03T18:40:10.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:09 smithi032 sudo[46929]: pam_unix(sudo:session): session closed for user root 2024-05-03T18:40:10.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:09 smithi032 sudo[46932]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-03T18:40:10.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:09 smithi032 sudo[46932]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-03T18:40:10.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:09 smithi032 sudo[46932]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-03T18:40:10.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:09 smithi032 sudo[46932]: pam_unix(sudo:session): session closed for user root 2024-05-03T18:40:10.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:40:10.233+0000 7f51d6c5e700 1 -- 172.21.15.32:0/259138957 --> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi032:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f51d00638e0 con 0x7f51b80b2870 2024-05-03T18:40:10.247 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:09 smithi045 sudo[42032]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-03T18:40:10.247 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:09 smithi045 sudo[42032]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-03T18:40:10.247 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:09 smithi045 sudo[42032]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-03T18:40:10.247 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:09 smithi045 sudo[42032]: pam_unix(sudo:session): session closed for user root 2024-05-03T18:40:10.247 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:09 smithi045 sudo[42035]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-03T18:40:10.247 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:09 smithi045 sudo[42035]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-03T18:40:10.247 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:09 smithi045 sudo[42035]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-03T18:40:10.248 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:09 smithi045 sudo[42035]: pam_unix(sudo:session): session closed for user root 2024-05-03T18:40:10.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:09 smithi113 sudo[42773]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-03T18:40:10.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:09 smithi113 sudo[42773]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-03T18:40:10.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:09 smithi113 sudo[42773]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-03T18:40:10.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:09 smithi113 sudo[42773]: pam_unix(sudo:session): session closed for user root 2024-05-03T18:40:10.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:09 smithi113 sudo[42776]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-03T18:40:10.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:09 smithi113 sudo[42776]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-03T18:40:10.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:09 smithi113 sudo[42776]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-03T18:40:10.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:09 smithi113 sudo[42776]: pam_unix(sudo:session): session closed for user root 2024-05-03T18:40:10.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:40:10.860+0000 7f51cdffb700 1 -- 172.21.15.32:0/259138957 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f51c00a5a20 con 0x7f51d0103540 2024-05-03T18:40:10.864 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:10 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-03T18:40:10.865 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:10 smithi032 ceph-mon[29763]: osdmap e20: 3 total, 3 up, 3 in 2024-05-03T18:40:10.865 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:10 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:40:10.865 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:10 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:40:10.865 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:10 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:40:10.865 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:10 smithi032 ceph-mon[29763]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-03T18:40:10.865 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:10 smithi032 ceph-mon[29763]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-03T18:40:10.865 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:10 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-03T18:40:10.865 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:10 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-03T18:40:10.865 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:10 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-03T18:40:10.865 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:10 smithi032 ceph-mon[29763]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-03T18:40:10.865 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:10 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-03T18:40:10.865 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:10 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-03T18:40:10.865 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:10 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-03T18:40:10.865 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:10 smithi032 ceph-mon[29763]: pgmap v82: 1 pgs: 1 active+clean; 320 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-03T18:40:10.865 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:10 smithi032 ceph-mon[29763]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-03T18:40:10.866 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:10 smithi032 ceph-mon[29763]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-03T18:40:10.866 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:10 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-03T18:40:10.866 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:10 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-03T18:40:10.866 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:10 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-03T18:40:10.866 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:10 smithi032 ceph-mon[29763]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-03T18:40:10.866 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:10 smithi032 ceph-mon[29763]: from='client.14310 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi032:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:40:10.866 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:10 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T18:40:10.866 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:10 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T18:40:10.866 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:10 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:40:10.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:10 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-03T18:40:10.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:10 smithi045 ceph-mon[33299]: osdmap e20: 3 total, 3 up, 3 in 2024-05-03T18:40:10.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:10 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:40:10.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:10 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:40:10.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:10 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:40:10.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:10 smithi045 ceph-mon[33299]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-03T18:40:10.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:10 smithi045 ceph-mon[33299]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-03T18:40:10.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:10 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-03T18:40:10.999 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:10 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-03T18:40:10.999 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:10 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-03T18:40:10.999 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:10 smithi045 ceph-mon[33299]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-03T18:40:10.999 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:10 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-03T18:40:10.999 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:10 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-03T18:40:10.999 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:10 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-03T18:40:10.999 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:10 smithi045 ceph-mon[33299]: pgmap v82: 1 pgs: 1 active+clean; 320 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-03T18:40:10.999 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:10 smithi045 ceph-mon[33299]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-03T18:40:10.999 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:10 smithi045 ceph-mon[33299]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-03T18:40:10.999 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:10 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-03T18:40:10.999 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:10 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-03T18:40:10.999 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:10 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-03T18:40:10.999 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:10 smithi045 ceph-mon[33299]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-03T18:40:11.000 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:10 smithi045 ceph-mon[33299]: from='client.14310 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi032:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:40:11.000 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:10 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T18:40:11.000 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:10 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T18:40:11.000 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:10 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:40:11.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:10 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-03T18:40:11.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:10 smithi113 ceph-mon[33540]: osdmap e20: 3 total, 3 up, 3 in 2024-05-03T18:40:11.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:10 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:40:11.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:10 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:40:11.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:10 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:40:11.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:10 smithi113 ceph-mon[33540]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-03T18:40:11.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:10 smithi113 ceph-mon[33540]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-03T18:40:11.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:10 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-03T18:40:11.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:10 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-03T18:40:11.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:10 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-03T18:40:11.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:10 smithi113 ceph-mon[33540]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-03T18:40:11.052 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:10 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-03T18:40:11.052 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:10 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-03T18:40:11.052 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:10 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-03T18:40:11.052 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:10 smithi113 ceph-mon[33540]: pgmap v82: 1 pgs: 1 active+clean; 320 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-03T18:40:11.052 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:10 smithi113 ceph-mon[33540]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-03T18:40:11.052 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:10 smithi113 ceph-mon[33540]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-03T18:40:11.052 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:10 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-03T18:40:11.052 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:10 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-03T18:40:11.052 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:10 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-03T18:40:11.052 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:10 smithi113 ceph-mon[33540]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-03T18:40:11.052 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:10 smithi113 ceph-mon[33540]: from='client.14310 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi032:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:40:11.052 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:10 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T18:40:11.052 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:10 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T18:40:11.052 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:10 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:40:11.997 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:11 smithi045 ceph-mon[33299]: osdmap e21: 3 total, 3 up, 3 in 2024-05-03T18:40:11.997 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:11 smithi045 ceph-mon[33299]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-05-03T18:40:12.029 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:11 smithi032 ceph-mon[29763]: osdmap e21: 3 total, 3 up, 3 in 2024-05-03T18:40:12.029 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:11 smithi032 ceph-mon[29763]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-05-03T18:40:12.050 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:11 smithi113 ceph-mon[33540]: osdmap e21: 3 total, 3 up, 3 in 2024-05-03T18:40:12.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:11 smithi113 ceph-mon[33540]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-05-03T18:40:12.997 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:12 smithi045 ceph-mon[33299]: pgmap v84: 1 pgs: 1 active+clean; 320 KiB data, 89 MiB used, 268 GiB / 268 GiB avail 2024-05-03T18:40:13.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:12 smithi113 ceph-mon[33540]: pgmap v84: 1 pgs: 1 active+clean; 320 KiB data, 89 MiB used, 268 GiB / 268 GiB avail 2024-05-03T18:40:13.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:12 smithi032 ceph-mon[29763]: pgmap v84: 1 pgs: 1 active+clean; 320 KiB data, 89 MiB used, 268 GiB / 268 GiB avail 2024-05-03T18:40:13.997 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:13 smithi045 ceph-mon[33299]: from='client.? 172.21.15.32:0/1117530232' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "62c9c977-afe1-4112-beba-b25af7caa2f7"}]: dispatch 2024-05-03T18:40:13.997 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:13 smithi045 ceph-mon[33299]: from='client.? 172.21.15.32:0/1117530232' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "62c9c977-afe1-4112-beba-b25af7caa2f7"}]': finished 2024-05-03T18:40:13.997 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:13 smithi045 ceph-mon[33299]: osdmap e22: 4 total, 3 up, 4 in 2024-05-03T18:40:13.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:13 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-03T18:40:14.050 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:13 smithi113 ceph-mon[33540]: from='client.? 172.21.15.32:0/1117530232' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "62c9c977-afe1-4112-beba-b25af7caa2f7"}]: dispatch 2024-05-03T18:40:14.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:13 smithi113 ceph-mon[33540]: from='client.? 172.21.15.32:0/1117530232' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "62c9c977-afe1-4112-beba-b25af7caa2f7"}]': finished 2024-05-03T18:40:14.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:13 smithi113 ceph-mon[33540]: osdmap e22: 4 total, 3 up, 4 in 2024-05-03T18:40:14.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:13 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-03T18:40:14.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:13 smithi032 ceph-mon[29763]: from='client.? 172.21.15.32:0/1117530232' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "62c9c977-afe1-4112-beba-b25af7caa2f7"}]: dispatch 2024-05-03T18:40:14.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:13 smithi032 ceph-mon[29763]: from='client.? 172.21.15.32:0/1117530232' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "62c9c977-afe1-4112-beba-b25af7caa2f7"}]': finished 2024-05-03T18:40:14.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:13 smithi032 ceph-mon[29763]: osdmap e22: 4 total, 3 up, 4 in 2024-05-03T18:40:14.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:13 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-03T18:40:14.997 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:14 smithi045 ceph-mon[33299]: from='client.? 172.21.15.32:0/4026149703' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T18:40:14.997 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:14 smithi045 ceph-mon[33299]: pgmap v86: 1 pgs: 1 active+clean; 320 KiB data, 89 MiB used, 268 GiB / 268 GiB avail 2024-05-03T18:40:15.050 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:14 smithi113 ceph-mon[33540]: from='client.? 172.21.15.32:0/4026149703' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T18:40:15.050 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:14 smithi113 ceph-mon[33540]: pgmap v86: 1 pgs: 1 active+clean; 320 KiB data, 89 MiB used, 268 GiB / 268 GiB avail 2024-05-03T18:40:15.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:14 smithi032 ceph-mon[29763]: from='client.? 172.21.15.32:0/4026149703' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T18:40:15.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:14 smithi032 ceph-mon[29763]: pgmap v86: 1 pgs: 1 active+clean; 320 KiB data, 89 MiB used, 268 GiB / 268 GiB avail 2024-05-03T18:40:17.247 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:16 smithi045 ceph-mon[33299]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 6.0 KiB/s rd, 305 KiB/s wr, 20 op/s 2024-05-03T18:40:17.300 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:16 smithi113 ceph-mon[33540]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 6.0 KiB/s rd, 305 KiB/s wr, 20 op/s 2024-05-03T18:40:17.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:16 smithi032 ceph-mon[29763]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 6.0 KiB/s rd, 305 KiB/s wr, 20 op/s 2024-05-03T18:40:19.201 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:18 smithi032 ceph-mon[29763]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 4.5 KiB/s rd, 228 KiB/s wr, 15 op/s 2024-05-03T18:40:19.247 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:18 smithi045 ceph-mon[33299]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 4.5 KiB/s rd, 228 KiB/s wr, 15 op/s 2024-05-03T18:40:19.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:18 smithi113 ceph-mon[33540]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 4.5 KiB/s rd, 228 KiB/s wr, 15 op/s 2024-05-03T18:40:21.247 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:20 smithi045 ceph-mon[33299]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 3.9 KiB/s rd, 197 KiB/s wr, 13 op/s 2024-05-03T18:40:21.248 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:20 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-03T18:40:21.248 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:20 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:40:21.248 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:20 smithi045 ceph-mon[33299]: Deploying daemon osd.3 on smithi032 2024-05-03T18:40:21.279 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:20 smithi032 ceph-mon[29763]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 3.9 KiB/s rd, 197 KiB/s wr, 13 op/s 2024-05-03T18:40:21.279 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:20 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-03T18:40:21.279 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:20 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:40:21.279 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:20 smithi032 ceph-mon[29763]: Deploying daemon osd.3 on smithi032 2024-05-03T18:40:21.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:20 smithi113 ceph-mon[33540]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 3.9 KiB/s rd, 197 KiB/s wr, 13 op/s 2024-05-03T18:40:21.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:20 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-03T18:40:21.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:20 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:40:21.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:20 smithi113 ceph-mon[33540]: Deploying daemon osd.3 on smithi032 2024-05-03T18:40:23.247 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:22 smithi045 ceph-mon[33299]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 3.6 KiB/s rd, 183 KiB/s wr, 12 op/s 2024-05-03T18:40:23.265 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:22 smithi032 ceph-mon[29763]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 3.6 KiB/s rd, 183 KiB/s wr, 12 op/s 2024-05-03T18:40:23.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:22 smithi113 ceph-mon[33540]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 3.6 KiB/s rd, 183 KiB/s wr, 12 op/s 2024-05-03T18:40:25.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:24 smithi032 ceph-mon[29763]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 3.3 KiB/s rd, 167 KiB/s wr, 11 op/s 2024-05-03T18:40:25.247 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:24 smithi045 ceph-mon[33299]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 3.3 KiB/s rd, 167 KiB/s wr, 11 op/s 2024-05-03T18:40:25.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:24 smithi113 ceph-mon[33540]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 3.3 KiB/s rd, 167 KiB/s wr, 11 op/s 2024-05-03T18:40:27.234 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:26 smithi032 ceph-mon[29763]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 152 KiB/s wr, 10 op/s 2024-05-03T18:40:27.235 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:26 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:40:27.235 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:26 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:40:27.235 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:26 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:40:27.247 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:26 smithi045 ceph-mon[33299]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 152 KiB/s wr, 10 op/s 2024-05-03T18:40:27.247 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:26 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:40:27.247 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:26 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:40:27.247 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:26 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:40:27.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:26 smithi113 ceph-mon[33540]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 152 KiB/s wr, 10 op/s 2024-05-03T18:40:27.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:26 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:40:27.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:26 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:40:27.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:26 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:40:29.236 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:28 smithi032 ceph-mon[29763]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-03T18:40:29.236 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:28 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:40:29.236 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:28 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:40:29.236 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:28 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:40:29.236 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:28 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:40:29.236 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:28 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:40:29.247 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:28 smithi045 ceph-mon[33299]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-03T18:40:29.247 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:28 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:40:29.247 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:28 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:40:29.247 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:28 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:40:29.248 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:28 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:40:29.248 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:28 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:40:29.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:28 smithi113 ceph-mon[33540]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-03T18:40:29.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:28 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:40:29.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:28 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:40:29.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:28 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:40:29.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:28 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:40:29.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:28 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:40:29.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:40:29.588+0000 7f51cdffb700 1 -- 172.21.15.32:0/259138957 <== mgr.14152 v2:172.21.15.32:6800/1796547904 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f51d00638e0 con 0x7f51b80b2870 2024-05-03T18:40:29.590 INFO:teuthology.orchestra.run.smithi032.stdout:Created osd(s) 3 on host 'smithi032' 2024-05-03T18:40:29.591 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:40:29.590+0000 7f51d6c5e700 1 -- 172.21.15.32:0/259138957 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f51b80b2870 msgr2=0x7f51b80b4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:40:29.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:40:29.590+0000 7f51d6c5e700 1 --2- 172.21.15.32:0/259138957 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f51b80b2870 0x7f51b80b4d30 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f51c4000c00 tx=0x7f51c4005fd0 comp rx=0 tx=0).stop 2024-05-03T18:40:29.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:40:29.590+0000 7f51d6c5e700 1 -- 172.21.15.32:0/259138957 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51d0103540 msgr2=0x7f51d0077240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:40:29.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:40:29.590+0000 7f51d6c5e700 1 --2- 172.21.15.32:0/259138957 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51d0103540 0x7f51d0077240 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f51c000c970 tx=0x7f51c000cc80 comp rx=0 tx=0).stop 2024-05-03T18:40:29.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:40:29.591+0000 7f51d6c5e700 1 -- 172.21.15.32:0/259138957 shutdown_connections 2024-05-03T18:40:29.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:40:29.591+0000 7f51d6c5e700 1 --2- 172.21.15.32:0/259138957 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f51b80b2870 0x7f51b80b4d30 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:29.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:40:29.591+0000 7f51d6c5e700 1 --2- 172.21.15.32:0/259138957 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f51d01050e0 0x7f51d0075dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:29.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:40:29.591+0000 7f51d6c5e700 1 --2- 172.21.15.32:0/259138957 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51d0103540 0x7f51d0077240 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:29.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:40:29.591+0000 7f51d6c5e700 1 --2- 172.21.15.32:0/259138957 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f51d0104740 0x7f51d0075890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:29.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:40:29.591+0000 7f51d6c5e700 1 -- 172.21.15.32:0/259138957 >> 172.21.15.32:0/259138957 conn(0x7f51d00feae0 msgr2=0x7f51d0107a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:40:29.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:40:29.591+0000 7f51d6c5e700 1 -- 172.21.15.32:0/259138957 shutdown_connections 2024-05-03T18:40:29.592 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:40:29.591+0000 7f51d6c5e700 1 -- 172.21.15.32:0/259138957 wait complete. 2024-05-03T18:40:30.118 DEBUG:teuthology.orchestra.run.smithi032:osd.3> sudo journalctl -f -n 0 -u ceph-f500455e-097b-11ef-bc97-c7b262605968@osd.3.service 2024-05-03T18:40:30.121 INFO:tasks.cephadm:Deploying osd.4 on smithi045 with /dev/vg_nvme/lv_3... 2024-05-03T18:40:30.121 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f500455e-097b-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-03T18:40:30.377 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/mon.b/config 2024-05-03T18:40:30.774 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:30 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:40:30.774 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:30 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:40:30.774 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:30 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:40:30.774 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:30 smithi045 ceph-mon[33299]: from='osd.3 [v2:172.21.15.32:6810/2606514063,v1:172.21.15.32:6811/2606514063]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-03T18:40:30.774 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:30 smithi045 ceph-mon[33299]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-03T18:40:30.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:30 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:40:30.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:30 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:40:30.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:30 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:40:30.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:30 smithi113 ceph-mon[33540]: from='osd.3 [v2:172.21.15.32:6810/2606514063,v1:172.21.15.32:6811/2606514063]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-03T18:40:30.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:30 smithi113 ceph-mon[33540]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-03T18:40:30.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:30 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:40:30.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:30 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:40:30.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:30 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:40:30.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:30 smithi032 ceph-mon[29763]: from='osd.3 [v2:172.21.15.32:6810/2606514063,v1:172.21.15.32:6811/2606514063]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-03T18:40:30.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:30 smithi032 ceph-mon[29763]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-03T18:40:31.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:31 smithi032 ceph-mon[29763]: from='osd.3 [v2:172.21.15.32:6810/2606514063,v1:172.21.15.32:6811/2606514063]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-03T18:40:31.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:31 smithi032 ceph-mon[29763]: osdmap e23: 4 total, 3 up, 4 in 2024-05-03T18:40:31.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:31 smithi032 ceph-mon[29763]: from='osd.3 [v2:172.21.15.32:6810/2606514063,v1:172.21.15.32:6811/2606514063]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi032", "root=default"]}]: dispatch 2024-05-03T18:40:31.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:31 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-03T18:40:31.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:31 smithi045 ceph-mon[33299]: from='osd.3 [v2:172.21.15.32:6810/2606514063,v1:172.21.15.32:6811/2606514063]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-03T18:40:31.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:31 smithi045 ceph-mon[33299]: osdmap e23: 4 total, 3 up, 4 in 2024-05-03T18:40:31.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:31 smithi045 ceph-mon[33299]: from='osd.3 [v2:172.21.15.32:6810/2606514063,v1:172.21.15.32:6811/2606514063]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi032", "root=default"]}]: dispatch 2024-05-03T18:40:31.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:31 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-03T18:40:32.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:31 smithi113 ceph-mon[33540]: from='osd.3 [v2:172.21.15.32:6810/2606514063,v1:172.21.15.32:6811/2606514063]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-03T18:40:32.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:31 smithi113 ceph-mon[33540]: osdmap e23: 4 total, 3 up, 4 in 2024-05-03T18:40:32.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:31 smithi113 ceph-mon[33540]: from='osd.3 [v2:172.21.15.32:6810/2606514063,v1:172.21.15.32:6811/2606514063]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi032", "root=default"]}]: dispatch 2024-05-03T18:40:32.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:31 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-03T18:40:32.102 INFO:teuthology.orchestra.run.smithi045.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-03T18:40:32.102 INFO:teuthology.orchestra.run.smithi045.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-03T18:40:32.102 INFO:teuthology.orchestra.run.smithi045.stderr: stderr: 10+0 records in 2024-05-03T18:40:32.102 INFO:teuthology.orchestra.run.smithi045.stderr:10+0 records out 2024-05-03T18:40:32.102 INFO:teuthology.orchestra.run.smithi045.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00824917 s, 1.3 GB/s 2024-05-03T18:40:32.102 INFO:teuthology.orchestra.run.smithi045.stderr:--> Zapping successful for: 2024-05-03T18:40:32.712 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f500455e-097b-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi045:vg_nvme/lv_3 2024-05-03T18:40:32.780 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:32 smithi032 ceph-mon[29763]: from='osd.3 [v2:172.21.15.32:6810/2606514063,v1:172.21.15.32:6811/2606514063]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi032", "root=default"]}]': finished 2024-05-03T18:40:32.780 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:32 smithi032 ceph-mon[29763]: osdmap e24: 4 total, 3 up, 4 in 2024-05-03T18:40:32.780 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:32 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-03T18:40:32.780 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:32 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-03T18:40:32.780 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:32 smithi032 ceph-mon[29763]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-03T18:40:32.781 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:32 smithi032 ceph-mon[29763]: from='osd.3 [v2:172.21.15.32:6810/2606514063,v1:172.21.15.32:6811/2606514063]' entity='osd.3' 2024-05-03T18:40:32.781 INFO:journalctl@ceph.osd.3.smithi032.stdout:May 03 18:40:32 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-3[49213]: 2024-05-03T18:40:32.338+0000 7f5855c08700 -1 osd.3 0 waiting for initial osdmap 2024-05-03T18:40:32.781 INFO:journalctl@ceph.osd.3.smithi032.stdout:May 03 18:40:32 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-3[49213]: 2024-05-03T18:40:32.343+0000 7f584d9f3700 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-03T18:40:32.955 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/mon.b/config 2024-05-03T18:40:32.997 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:32 smithi045 ceph-mon[33299]: from='osd.3 [v2:172.21.15.32:6810/2606514063,v1:172.21.15.32:6811/2606514063]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi032", "root=default"]}]': finished 2024-05-03T18:40:32.997 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:32 smithi045 ceph-mon[33299]: osdmap e24: 4 total, 3 up, 4 in 2024-05-03T18:40:32.997 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:32 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-03T18:40:32.997 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:32 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-03T18:40:32.997 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:32 smithi045 ceph-mon[33299]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-03T18:40:32.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:32 smithi045 ceph-mon[33299]: from='osd.3 [v2:172.21.15.32:6810/2606514063,v1:172.21.15.32:6811/2606514063]' entity='osd.3' 2024-05-03T18:40:33.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:32 smithi113 ceph-mon[33540]: from='osd.3 [v2:172.21.15.32:6810/2606514063,v1:172.21.15.32:6811/2606514063]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi032", "root=default"]}]': finished 2024-05-03T18:40:33.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:32 smithi113 ceph-mon[33540]: osdmap e24: 4 total, 3 up, 4 in 2024-05-03T18:40:33.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:32 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-03T18:40:33.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:32 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-03T18:40:33.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:32 smithi113 ceph-mon[33540]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-05-03T18:40:33.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:32 smithi113 ceph-mon[33540]: from='osd.3 [v2:172.21.15.32:6810/2606514063,v1:172.21.15.32:6811/2606514063]' entity='osd.3' 2024-05-03T18:40:33.747 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:33 smithi045 ceph-mon[33299]: purged_snaps scrub starts 2024-05-03T18:40:33.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:33 smithi045 ceph-mon[33299]: purged_snaps scrub ok 2024-05-03T18:40:33.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:33 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-03T18:40:33.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:33 smithi045 ceph-mon[33299]: osd.3 [v2:172.21.15.32:6810/2606514063,v1:172.21.15.32:6811/2606514063] boot 2024-05-03T18:40:33.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:33 smithi045 ceph-mon[33299]: osdmap e25: 4 total, 4 up, 4 in 2024-05-03T18:40:33.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:33 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-03T18:40:33.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:33 smithi045 ceph-mon[33299]: Detected new or changed devices on smithi032 2024-05-03T18:40:33.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:33 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:40:33.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:33 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:40:33.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:33 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:40:33.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:33 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:40:33.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:33 smithi045 ceph-mon[33299]: Adjusting osd_memory_target on smithi032 to 8546M 2024-05-03T18:40:33.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:33 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:40:33.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:33 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:40:33.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:33 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:40:33.749 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:33 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:40:33.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:33 smithi032 ceph-mon[29763]: purged_snaps scrub starts 2024-05-03T18:40:33.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:33 smithi032 ceph-mon[29763]: purged_snaps scrub ok 2024-05-03T18:40:33.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:33 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-03T18:40:33.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:33 smithi032 ceph-mon[29763]: osd.3 [v2:172.21.15.32:6810/2606514063,v1:172.21.15.32:6811/2606514063] boot 2024-05-03T18:40:33.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:33 smithi032 ceph-mon[29763]: osdmap e25: 4 total, 4 up, 4 in 2024-05-03T18:40:33.819 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:33 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-03T18:40:33.819 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:33 smithi032 ceph-mon[29763]: Detected new or changed devices on smithi032 2024-05-03T18:40:33.819 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:33 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:40:33.819 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:33 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:40:33.819 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:33 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:40:33.819 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:33 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:40:33.819 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:33 smithi032 ceph-mon[29763]: Adjusting osd_memory_target on smithi032 to 8546M 2024-05-03T18:40:33.819 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:33 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:40:33.819 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:33 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:40:33.819 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:33 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:40:33.819 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:33 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:40:34.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:33 smithi113 ceph-mon[33540]: purged_snaps scrub starts 2024-05-03T18:40:34.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:33 smithi113 ceph-mon[33540]: purged_snaps scrub ok 2024-05-03T18:40:34.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:33 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-03T18:40:34.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:33 smithi113 ceph-mon[33540]: osd.3 [v2:172.21.15.32:6810/2606514063,v1:172.21.15.32:6811/2606514063] boot 2024-05-03T18:40:34.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:33 smithi113 ceph-mon[33540]: osdmap e25: 4 total, 4 up, 4 in 2024-05-03T18:40:34.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:33 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-03T18:40:34.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:33 smithi113 ceph-mon[33540]: Detected new or changed devices on smithi032 2024-05-03T18:40:34.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:33 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:40:34.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:33 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:40:34.052 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:33 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:40:34.052 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:33 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:40:34.052 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:33 smithi113 ceph-mon[33540]: Adjusting osd_memory_target on smithi032 to 8546M 2024-05-03T18:40:34.052 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:33 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:40:34.052 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:33 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:40:34.052 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:33 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:40:34.052 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:33 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:40:34.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:40:34.605+0000 7f05a7f63700 1 -- 172.21.15.45:0/3400111233 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05a0104700 msgr2=0x7f05a0104b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:40:34.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:40:34.605+0000 7f05a7f63700 1 --2- 172.21.15.45:0/3400111233 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05a0104700 0x7f05a0104b60 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f05900099c0 tx=0x7f0590009cd0 comp rx=0 tx=0).stop 2024-05-03T18:40:34.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:40:34.605+0000 7f05a7f63700 1 -- 172.21.15.45:0/3400111233 shutdown_connections 2024-05-03T18:40:34.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:40:34.605+0000 7f05a7f63700 1 --2- 172.21.15.45:0/3400111233 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f05a01050a0 0x7f05a010b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:34.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:40:34.605+0000 7f05a7f63700 1 --2- 172.21.15.45:0/3400111233 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05a0103500 0x7f05a0103920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:34.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:40:34.605+0000 7f05a7f63700 1 --2- 172.21.15.45:0/3400111233 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05a0104700 0x7f05a0104b60 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:34.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:40:34.605+0000 7f05a7f63700 1 -- 172.21.15.45:0/3400111233 >> 172.21.15.45:0/3400111233 conn(0x7f05a00feaa0 msgr2=0x7f05a0100ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:40:34.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:40:34.605+0000 7f05a7f63700 1 -- 172.21.15.45:0/3400111233 shutdown_connections 2024-05-03T18:40:34.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:40:34.605+0000 7f05a7f63700 1 -- 172.21.15.45:0/3400111233 wait complete. 2024-05-03T18:40:34.608 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:40:34.606+0000 7f05a7f63700 1 Processor -- start 2024-05-03T18:40:34.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:40:34.606+0000 7f05a7f63700 1 -- start start 2024-05-03T18:40:34.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:40:34.606+0000 7f05a7f63700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f05a0103500 0x7f05a019b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:40:34.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:40:34.606+0000 7f05a7f63700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05a0104700 0x7f05a019bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:40:34.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:40:34.606+0000 7f05a7f63700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05a01050a0 0x7f05a01a2e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:40:34.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:40:34.606+0000 7f05a7f63700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f05a010eff0 con 0x7f05a01050a0 2024-05-03T18:40:34.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:40:34.606+0000 7f05a7f63700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f05a010ee70 con 0x7f05a0104700 2024-05-03T18:40:34.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:40:34.606+0000 7f05a7f63700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f05a010f170 con 0x7f05a0103500 2024-05-03T18:40:34.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:40:34.606+0000 7f05a54fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05a0104700 0x7f05a019bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:40:34.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:40:34.606+0000 7f05a54fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05a0104700 0x7f05a019bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52598/0 (socket says 172.21.15.45:52598) 2024-05-03T18:40:34.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:40:34.606+0000 7f05a54fe700 1 -- 172.21.15.45:0/3002736524 learned_addr learned my addr 172.21.15.45:0/3002736524 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-03T18:40:34.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:40:34.606+0000 7f05a5cff700 1 --2- 172.21.15.45:0/3002736524 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f05a0103500 0x7f05a019b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:40:34.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:40:34.606+0000 7f05a6500700 1 --2- 172.21.15.45:0/3002736524 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05a01050a0 0x7f05a01a2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:40:34.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:40:34.607+0000 7f05a54fe700 1 -- 172.21.15.45:0/3002736524 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f05a0103500 msgr2=0x7f05a019b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:40:34.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:40:34.607+0000 7f05a54fe700 1 --2- 172.21.15.45:0/3002736524 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f05a0103500 0x7f05a019b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:34.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:40:34.607+0000 7f05a54fe700 1 -- 172.21.15.45:0/3002736524 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05a01050a0 msgr2=0x7f05a01a2e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:40:34.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:40:34.607+0000 7f05a54fe700 1 --2- 172.21.15.45:0/3002736524 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05a01050a0 0x7f05a01a2e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:34.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:40:34.607+0000 7f05a54fe700 1 -- 172.21.15.45:0/3002736524 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05a01a3510 con 0x7f05a0104700 2024-05-03T18:40:34.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:40:34.607+0000 7f05a54fe700 1 --2- 172.21.15.45:0/3002736524 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05a0104700 0x7f05a019bdf0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f0590003790 tx=0x7f0590005ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:40:34.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:40:34.607+0000 7f0596ffd700 1 -- 172.21.15.45:0/3002736524 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0590027070 con 0x7f05a0104700 2024-05-03T18:40:34.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:40:34.607+0000 7f0596ffd700 1 -- 172.21.15.45:0/3002736524 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0590022470 con 0x7f05a0104700 2024-05-03T18:40:34.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:40:34.607+0000 7f05a7f63700 1 -- 172.21.15.45:0/3002736524 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05a01a37a0 con 0x7f05a0104700 2024-05-03T18:40:34.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:40:34.607+0000 7f0596ffd700 1 -- 172.21.15.45:0/3002736524 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f059000f6c0 con 0x7f05a0104700 2024-05-03T18:40:34.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:40:34.607+0000 7f05a7f63700 1 -- 172.21.15.45:0/3002736524 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05a01a3cd0 con 0x7f05a0104700 2024-05-03T18:40:34.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:40:34.608+0000 7f05a7f63700 1 -- 172.21.15.45:0/3002736524 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05a006a190 con 0x7f05a0104700 2024-05-03T18:40:34.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:40:34.610+0000 7f0596ffd700 1 -- 172.21.15.45:0/3002736524 <== mon.2 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f059002a070 con 0x7f05a0104700 2024-05-03T18:40:34.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:40:34.611+0000 7f0596ffd700 1 --2- 172.21.15.45:0/3002736524 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f058c0b2900 0x7f058c0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:40:34.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:40:34.611+0000 7f0596ffd700 1 -- 172.21.15.45:0/3002736524 <== mon.2 v2:172.21.15.45:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 3669+0+0 (secure 0 0 0) 0x7f05900e1fe0 con 0x7f05a0104700 2024-05-03T18:40:34.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:40:34.611+0000 7f05a5cff700 1 --2- 172.21.15.45:0/3002736524 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f058c0b2900 0x7f058c0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:40:34.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:40:34.612+0000 7f05a5cff700 1 --2- 172.21.15.45:0/3002736524 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f058c0b2900 0x7f058c0b4dc0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f05a0104560 tx=0x7f059c00a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:40:34.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:40:34.615+0000 7f0596ffd700 1 -- 172.21.15.45:0/3002736524 <== mon.2 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f0590013070 con 0x7f05a0104700 2024-05-03T18:40:34.747 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:34 smithi045 ceph-mon[33299]: osdmap e26: 4 total, 4 up, 4 in 2024-05-03T18:40:34.747 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:34 smithi045 ceph-mon[33299]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail 2024-05-03T18:40:34.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:40:34.842+0000 7f05a7f63700 1 -- 172.21.15.45:0/3002736524 --> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f05a0076df0 con 0x7f058c0b2900 2024-05-03T18:40:35.050 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:34 smithi113 ceph-mon[33540]: osdmap e26: 4 total, 4 up, 4 in 2024-05-03T18:40:35.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:34 smithi113 ceph-mon[33540]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail 2024-05-03T18:40:35.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:34 smithi032 ceph-mon[29763]: osdmap e26: 4 total, 4 up, 4 in 2024-05-03T18:40:35.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:34 smithi032 ceph-mon[29763]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail 2024-05-03T18:40:35.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:35 smithi045 ceph-mon[33299]: osdmap e27: 4 total, 4 up, 4 in 2024-05-03T18:40:35.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:35 smithi045 ceph-mon[33299]: from='client.24191 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:40:35.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:35 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T18:40:35.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:35 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T18:40:35.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:35 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:40:35.970 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:35 smithi032 ceph-mon[29763]: osdmap e27: 4 total, 4 up, 4 in 2024-05-03T18:40:35.970 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:35 smithi032 ceph-mon[29763]: from='client.24191 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:40:35.970 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:35 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T18:40:35.970 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:35 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T18:40:35.970 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:35 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:40:36.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:35 smithi113 ceph-mon[33540]: osdmap e27: 4 total, 4 up, 4 in 2024-05-03T18:40:36.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:35 smithi113 ceph-mon[33540]: from='client.24191 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:40:36.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:35 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T18:40:36.052 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:35 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T18:40:36.052 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:35 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:40:36.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:36 smithi113 ceph-mon[33540]: pgmap v102: 1 pgs: 1 activating+degraded; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-05-03T18:40:36.997 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:36 smithi045 ceph-mon[33299]: pgmap v102: 1 pgs: 1 activating+degraded; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-05-03T18:40:37.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:36 smithi032 ceph-mon[29763]: pgmap v102: 1 pgs: 1 activating+degraded; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-05-03T18:40:37.997 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:37 smithi045 ceph-mon[33299]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-05-03T18:40:37.997 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:37 smithi045 ceph-mon[33299]: from='client.? 172.21.15.45:0/1143454773' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "99ad449f-b0bc-4aa1-b0cf-4a7a458126e6"}]: dispatch 2024-05-03T18:40:37.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:37 smithi045 ceph-mon[33299]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "99ad449f-b0bc-4aa1-b0cf-4a7a458126e6"}]: dispatch 2024-05-03T18:40:37.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:37 smithi045 ceph-mon[33299]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "99ad449f-b0bc-4aa1-b0cf-4a7a458126e6"}]': finished 2024-05-03T18:40:37.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:37 smithi045 ceph-mon[33299]: osdmap e28: 5 total, 4 up, 5 in 2024-05-03T18:40:37.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:37 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-03T18:40:38.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:37 smithi113 ceph-mon[33540]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-05-03T18:40:38.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:37 smithi113 ceph-mon[33540]: from='client.? 172.21.15.45:0/1143454773' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "99ad449f-b0bc-4aa1-b0cf-4a7a458126e6"}]: dispatch 2024-05-03T18:40:38.052 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:37 smithi113 ceph-mon[33540]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "99ad449f-b0bc-4aa1-b0cf-4a7a458126e6"}]: dispatch 2024-05-03T18:40:38.052 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:37 smithi113 ceph-mon[33540]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "99ad449f-b0bc-4aa1-b0cf-4a7a458126e6"}]': finished 2024-05-03T18:40:38.052 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:37 smithi113 ceph-mon[33540]: osdmap e28: 5 total, 4 up, 5 in 2024-05-03T18:40:38.052 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:37 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-03T18:40:38.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:37 smithi032 ceph-mon[29763]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-05-03T18:40:38.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:37 smithi032 ceph-mon[29763]: from='client.? 172.21.15.45:0/1143454773' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "99ad449f-b0bc-4aa1-b0cf-4a7a458126e6"}]: dispatch 2024-05-03T18:40:38.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:37 smithi032 ceph-mon[29763]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "99ad449f-b0bc-4aa1-b0cf-4a7a458126e6"}]: dispatch 2024-05-03T18:40:38.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:37 smithi032 ceph-mon[29763]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "99ad449f-b0bc-4aa1-b0cf-4a7a458126e6"}]': finished 2024-05-03T18:40:38.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:37 smithi032 ceph-mon[29763]: osdmap e28: 5 total, 4 up, 5 in 2024-05-03T18:40:38.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:37 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-03T18:40:38.997 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:38 smithi045 ceph-mon[33299]: pgmap v104: 1 pgs: 1 activating+degraded; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-05-03T18:40:38.997 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:38 smithi045 ceph-mon[33299]: from='client.? 172.21.15.45:0/1537068232' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T18:40:39.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:38 smithi113 ceph-mon[33540]: pgmap v104: 1 pgs: 1 activating+degraded; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-05-03T18:40:39.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:38 smithi113 ceph-mon[33540]: from='client.? 172.21.15.45:0/1537068232' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T18:40:39.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:38 smithi032 ceph-mon[29763]: pgmap v104: 1 pgs: 1 activating+degraded; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-05-03T18:40:39.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:38 smithi032 ceph-mon[29763]: from='client.? 172.21.15.45:0/1537068232' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T18:40:40.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:40 smithi113 ceph-mon[33540]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-05-03T18:40:40.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:40 smithi113 ceph-mon[33540]: Cluster is now healthy 2024-05-03T18:40:40.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:40 smithi032 ceph-mon[29763]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-05-03T18:40:40.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:40 smithi032 ceph-mon[29763]: Cluster is now healthy 2024-05-03T18:40:40.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:40 smithi045 ceph-mon[33299]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-05-03T18:40:40.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:40 smithi045 ceph-mon[33299]: Cluster is now healthy 2024-05-03T18:40:41.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:41 smithi113 ceph-mon[33540]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 91 KiB/s, 0 objects/s recovering 2024-05-03T18:40:41.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:41 smithi032 ceph-mon[29763]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 91 KiB/s, 0 objects/s recovering 2024-05-03T18:40:41.424 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:41 smithi045 ceph-mon[33299]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 91 KiB/s, 0 objects/s recovering 2024-05-03T18:40:43.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:43 smithi113 ceph-mon[33540]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-03T18:40:43.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:43 smithi032 ceph-mon[29763]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-03T18:40:43.322 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:43 smithi045 ceph-mon[33299]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-03T18:40:45.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:45 smithi113 ceph-mon[33540]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-05-03T18:40:45.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:45 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-03T18:40:45.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:45 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:40:45.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:45 smithi032 ceph-mon[29763]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-05-03T18:40:45.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:45 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-03T18:40:45.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:45 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:40:45.343 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:45 smithi045 ceph-mon[33299]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-05-03T18:40:45.343 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:45 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-03T18:40:45.343 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:45 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:40:46.247 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:46 smithi045 ceph-mon[33299]: Deploying daemon osd.4 on smithi045 2024-05-03T18:40:46.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:46 smithi113 ceph-mon[33540]: Deploying daemon osd.4 on smithi045 2024-05-03T18:40:46.317 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:46 smithi032 ceph-mon[29763]: Deploying daemon osd.4 on smithi045 2024-05-03T18:40:47.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:47 smithi113 ceph-mon[33540]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-03T18:40:47.302 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:47 smithi032 ceph-mon[29763]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-03T18:40:47.357 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:47 smithi045 ceph-mon[33299]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-03T18:40:49.206 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:49 smithi045 ceph-mon[33299]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-05-03T18:40:49.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:49 smithi113 ceph-mon[33540]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-05-03T18:40:49.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:49 smithi032 ceph-mon[29763]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-05-03T18:40:51.247 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:51 smithi045 ceph-mon[33299]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-05-03T18:40:51.248 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:51 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:40:51.248 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:51 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:40:51.248 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:51 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:40:51.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:51 smithi113 ceph-mon[33540]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-05-03T18:40:51.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:51 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:40:51.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:51 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:40:51.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:51 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:40:51.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:51 smithi032 ceph-mon[29763]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-05-03T18:40:51.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:51 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:40:51.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:51 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:40:51.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:51 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:40:53.660 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:53 smithi045 ceph-mon[33299]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-05-03T18:40:53.660 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:53 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:40:53.660 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:53 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:40:53.660 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:53 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:40:53.660 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:53 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:40:53.661 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:53 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:40:53.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:40:53.711+0000 7f0596ffd700 1 -- 172.21.15.45:0/3002736524 <== mgr.14152 v2:172.21.15.32:6800/1796547904 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f05a0076df0 con 0x7f058c0b2900 2024-05-03T18:40:53.712 INFO:teuthology.orchestra.run.smithi045.stdout:Created osd(s) 4 on host 'smithi045' 2024-05-03T18:40:53.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:40:53.714+0000 7f05a7f63700 1 -- 172.21.15.45:0/3002736524 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f058c0b2900 msgr2=0x7f058c0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:40:53.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:40:53.714+0000 7f05a7f63700 1 --2- 172.21.15.45:0/3002736524 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f058c0b2900 0x7f058c0b4dc0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f05a0104560 tx=0x7f059c00a380 comp rx=0 tx=0).stop 2024-05-03T18:40:53.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:40:53.714+0000 7f05a7f63700 1 -- 172.21.15.45:0/3002736524 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05a0104700 msgr2=0x7f05a019bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:40:53.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:40:53.714+0000 7f05a7f63700 1 --2- 172.21.15.45:0/3002736524 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05a0104700 0x7f05a019bdf0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f0590003790 tx=0x7f0590005ef0 comp rx=0 tx=0).stop 2024-05-03T18:40:53.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:40:53.714+0000 7f05a7f63700 1 -- 172.21.15.45:0/3002736524 shutdown_connections 2024-05-03T18:40:53.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:40:53.714+0000 7f05a7f63700 1 --2- 172.21.15.45:0/3002736524 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f058c0b2900 0x7f058c0b4dc0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:53.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:40:53.714+0000 7f05a7f63700 1 --2- 172.21.15.45:0/3002736524 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05a01050a0 0x7f05a01a2e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:53.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:40:53.714+0000 7f05a7f63700 1 --2- 172.21.15.45:0/3002736524 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f05a0103500 0x7f05a019b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:53.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:40:53.714+0000 7f05a7f63700 1 --2- 172.21.15.45:0/3002736524 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05a0104700 0x7f05a019bdf0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:53.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:40:53.714+0000 7f05a7f63700 1 -- 172.21.15.45:0/3002736524 >> 172.21.15.45:0/3002736524 conn(0x7f05a00feaa0 msgr2=0x7f05a00756d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:40:53.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:40:53.714+0000 7f05a7f63700 1 -- 172.21.15.45:0/3002736524 shutdown_connections 2024-05-03T18:40:53.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:40:53.714+0000 7f05a7f63700 1 -- 172.21.15.45:0/3002736524 wait complete. 2024-05-03T18:40:53.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:53 smithi113 ceph-mon[33540]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-05-03T18:40:53.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:53 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:40:53.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:53 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:40:53.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:53 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:40:53.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:53 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:40:53.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:53 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:40:53.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:53 smithi032 ceph-mon[29763]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-05-03T18:40:53.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:53 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:40:53.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:53 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:40:53.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:53 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:40:53.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:53 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:40:53.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:53 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:40:54.365 DEBUG:teuthology.orchestra.run.smithi045:osd.4> sudo journalctl -f -n 0 -u ceph-f500455e-097b-11ef-bc97-c7b262605968@osd.4.service 2024-05-03T18:40:54.368 INFO:tasks.cephadm:Deploying osd.5 on smithi113 with /dev/vg_nvme/lv_3... 2024-05-03T18:40:54.368 DEBUG:teuthology.orchestra.run.smithi113:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f500455e-097b-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-03T18:40:54.611 INFO:teuthology.orchestra.run.smithi113.stderr:Inferring config /var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/mon.c/config 2024-05-03T18:40:54.946 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:54 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:40:54.946 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:54 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:40:54.946 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:54 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:40:54.946 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:54 smithi113 ceph-mon[33540]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-05-03T18:40:54.946 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:54 smithi113 ceph-mon[33540]: from='osd.4 [v2:172.21.15.45:6808/71932118,v1:172.21.15.45:6809/71932118]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-03T18:40:54.946 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:54 smithi113 ceph-mon[33540]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-03T18:40:54.997 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:54 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:40:54.997 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:54 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:40:54.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:54 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:40:54.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:54 smithi045 ceph-mon[33299]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-05-03T18:40:54.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:54 smithi045 ceph-mon[33299]: from='osd.4 [v2:172.21.15.45:6808/71932118,v1:172.21.15.45:6809/71932118]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-03T18:40:54.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:54 smithi045 ceph-mon[33299]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-03T18:40:55.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:54 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:40:55.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:54 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:40:55.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:54 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:40:55.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:54 smithi032 ceph-mon[29763]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-05-03T18:40:55.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:54 smithi032 ceph-mon[29763]: from='osd.4 [v2:172.21.15.45:6808/71932118,v1:172.21.15.45:6809/71932118]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-03T18:40:55.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:54 smithi032 ceph-mon[29763]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-03T18:40:55.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:55 smithi045 ceph-mon[33299]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-03T18:40:55.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:55 smithi045 ceph-mon[33299]: osdmap e29: 5 total, 4 up, 5 in 2024-05-03T18:40:55.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:55 smithi045 ceph-mon[33299]: from='osd.4 [v2:172.21.15.45:6808/71932118,v1:172.21.15.45:6809/71932118]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-05-03T18:40:55.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:55 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-03T18:40:55.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:55 smithi045 ceph-mon[33299]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-05-03T18:40:55.748 INFO:journalctl@ceph.osd.4.smithi045.stdout:May 03 18:40:55 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-4[44832]: 2024-05-03T18:40:55.713+0000 7fc83bbce700 -1 osd.4 0 waiting for initial osdmap 2024-05-03T18:40:55.748 INFO:journalctl@ceph.osd.4.smithi045.stdout:May 03 18:40:55 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-4[44832]: 2024-05-03T18:40:55.719+0000 7fc8379c1700 -1 osd.4 30 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-03T18:40:55.905 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:55 smithi113 ceph-mon[33540]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-03T18:40:55.905 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:55 smithi113 ceph-mon[33540]: osdmap e29: 5 total, 4 up, 5 in 2024-05-03T18:40:55.905 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:55 smithi113 ceph-mon[33540]: from='osd.4 [v2:172.21.15.45:6808/71932118,v1:172.21.15.45:6809/71932118]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-05-03T18:40:55.905 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:55 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-03T18:40:55.906 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:55 smithi113 ceph-mon[33540]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-05-03T18:40:56.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:55 smithi032 ceph-mon[29763]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-03T18:40:56.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:55 smithi032 ceph-mon[29763]: osdmap e29: 5 total, 4 up, 5 in 2024-05-03T18:40:56.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:55 smithi032 ceph-mon[29763]: from='osd.4 [v2:172.21.15.45:6808/71932118,v1:172.21.15.45:6809/71932118]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-05-03T18:40:56.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:55 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-03T18:40:56.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:55 smithi032 ceph-mon[29763]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-05-03T18:40:56.266 INFO:teuthology.orchestra.run.smithi113.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-03T18:40:56.266 INFO:teuthology.orchestra.run.smithi113.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-03T18:40:56.266 INFO:teuthology.orchestra.run.smithi113.stderr: stderr: 10+0 records in 2024-05-03T18:40:56.266 INFO:teuthology.orchestra.run.smithi113.stderr:10+0 records out 2024-05-03T18:40:56.266 INFO:teuthology.orchestra.run.smithi113.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00853839 s, 1.2 GB/s 2024-05-03T18:40:56.266 INFO:teuthology.orchestra.run.smithi113.stderr:--> Zapping successful for: 2024-05-03T18:40:56.925 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:56 smithi113 ceph-mon[33540]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2024-05-03T18:40:56.926 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:56 smithi113 ceph-mon[33540]: osdmap e30: 5 total, 4 up, 5 in 2024-05-03T18:40:56.926 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:56 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-03T18:40:56.926 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:56 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-03T18:40:56.926 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:56 smithi113 ceph-mon[33540]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-05-03T18:40:56.953 DEBUG:teuthology.orchestra.run.smithi113:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f500455e-097b-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi113:vg_nvme/lv_3 2024-05-03T18:40:56.997 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:56 smithi045 ceph-mon[33299]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2024-05-03T18:40:56.997 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:56 smithi045 ceph-mon[33299]: osdmap e30: 5 total, 4 up, 5 in 2024-05-03T18:40:56.997 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:56 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-03T18:40:56.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:56 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-03T18:40:56.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:56 smithi045 ceph-mon[33299]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-05-03T18:40:57.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:56 smithi032 ceph-mon[29763]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2024-05-03T18:40:57.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:56 smithi032 ceph-mon[29763]: osdmap e30: 5 total, 4 up, 5 in 2024-05-03T18:40:57.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:56 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-03T18:40:57.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:56 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-03T18:40:57.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:56 smithi032 ceph-mon[29763]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-05-03T18:40:57.190 INFO:teuthology.orchestra.run.smithi113.stderr:Inferring config /var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/mon.c/config 2024-05-03T18:40:57.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:57 smithi045 ceph-mon[33299]: purged_snaps scrub starts 2024-05-03T18:40:57.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:57 smithi045 ceph-mon[33299]: purged_snaps scrub ok 2024-05-03T18:40:57.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:57 smithi045 ceph-mon[33299]: osd.4 [v2:172.21.15.45:6808/71932118,v1:172.21.15.45:6809/71932118] boot 2024-05-03T18:40:57.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:57 smithi045 ceph-mon[33299]: osdmap e31: 5 total, 5 up, 5 in 2024-05-03T18:40:57.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:57 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-03T18:40:57.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:57 smithi045 ceph-mon[33299]: Detected new or changed devices on smithi045 2024-05-03T18:40:57.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:57 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:40:57.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:57 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:40:57.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:57 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:40:57.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:57 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:40:57.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:57 smithi045 ceph-mon[33299]: Adjusting osd_memory_target on smithi045 to 8546M 2024-05-03T18:40:57.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:57 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:40:57.999 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:57 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:40:57.999 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:57 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:40:57.999 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:57 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:40:58.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:57 smithi113 ceph-mon[33540]: purged_snaps scrub starts 2024-05-03T18:40:58.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:57 smithi113 ceph-mon[33540]: purged_snaps scrub ok 2024-05-03T18:40:58.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:57 smithi113 ceph-mon[33540]: osd.4 [v2:172.21.15.45:6808/71932118,v1:172.21.15.45:6809/71932118] boot 2024-05-03T18:40:58.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:57 smithi113 ceph-mon[33540]: osdmap e31: 5 total, 5 up, 5 in 2024-05-03T18:40:58.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:57 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-03T18:40:58.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:57 smithi113 ceph-mon[33540]: Detected new or changed devices on smithi045 2024-05-03T18:40:58.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:57 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:40:58.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:57 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:40:58.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:57 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:40:58.052 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:57 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:40:58.052 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:57 smithi113 ceph-mon[33540]: Adjusting osd_memory_target on smithi045 to 8546M 2024-05-03T18:40:58.052 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:57 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:40:58.052 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:57 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:40:58.052 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:57 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:40:58.052 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:57 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:40:58.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:57 smithi032 ceph-mon[29763]: purged_snaps scrub starts 2024-05-03T18:40:58.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:57 smithi032 ceph-mon[29763]: purged_snaps scrub ok 2024-05-03T18:40:58.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:57 smithi032 ceph-mon[29763]: osd.4 [v2:172.21.15.45:6808/71932118,v1:172.21.15.45:6809/71932118] boot 2024-05-03T18:40:58.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:57 smithi032 ceph-mon[29763]: osdmap e31: 5 total, 5 up, 5 in 2024-05-03T18:40:58.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:57 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-03T18:40:58.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:57 smithi032 ceph-mon[29763]: Detected new or changed devices on smithi045 2024-05-03T18:40:58.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:57 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:40:58.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:57 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:40:58.069 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:57 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:40:58.069 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:57 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:40:58.069 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:57 smithi032 ceph-mon[29763]: Adjusting osd_memory_target on smithi045 to 8546M 2024-05-03T18:40:58.069 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:57 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:40:58.069 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:57 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:40:58.069 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:57 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:40:58.069 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:57 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:40:58.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:40:58.662+0000 7f3d711dd700 1 -- 172.21.15.113:0/4004007738 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3d6c1017c0 msgr2=0x7f3d6c101be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:40:58.665 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:40:58.662+0000 7f3d711dd700 1 --2- 172.21.15.113:0/4004007738 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3d6c1017c0 0x7f3d6c101be0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f3d5c009990 tx=0x7f3d5c009ca0 comp rx=0 tx=0).stop 2024-05-03T18:40:58.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:40:58.662+0000 7f3d711dd700 1 -- 172.21.15.113:0/4004007738 shutdown_connections 2024-05-03T18:40:58.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:40:58.662+0000 7f3d711dd700 1 --2- 172.21.15.113:0/4004007738 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d6c105140 0x7f3d6c10b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:58.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:40:58.662+0000 7f3d711dd700 1 --2- 172.21.15.113:0/4004007738 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3d6c1017c0 0x7f3d6c101be0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:58.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:40:58.662+0000 7f3d711dd700 1 --2- 172.21.15.113:0/4004007738 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3d6c1047a0 0x7f3d6c104c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:58.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:40:58.662+0000 7f3d711dd700 1 -- 172.21.15.113:0/4004007738 >> 172.21.15.113:0/4004007738 conn(0x7f3d6c0fd100 msgr2=0x7f3d6c0ff560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:40:58.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:40:58.662+0000 7f3d711dd700 1 -- 172.21.15.113:0/4004007738 shutdown_connections 2024-05-03T18:40:58.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:40:58.662+0000 7f3d711dd700 1 -- 172.21.15.113:0/4004007738 wait complete. 2024-05-03T18:40:58.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:40:58.663+0000 7f3d711dd700 1 Processor -- start 2024-05-03T18:40:58.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:40:58.663+0000 7f3d711dd700 1 -- start start 2024-05-03T18:40:58.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:40:58.663+0000 7f3d711dd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3d6c1017c0 0x7f3d6c1040b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:40:58.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:40:58.663+0000 7f3d711dd700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3d6c1047a0 0x7f3d6c102750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:40:58.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:40:58.663+0000 7f3d711dd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d6c105140 0x7f3d6c102c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:40:58.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:40:58.663+0000 7f3d711dd700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f3d6c10efc0 con 0x7f3d6c1017c0 2024-05-03T18:40:58.666 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:40:58.663+0000 7f3d711dd700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3d6c10ee40 con 0x7f3d6c105140 2024-05-03T18:40:58.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:40:58.663+0000 7f3d711dd700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f3d6c10f140 con 0x7f3d6c1047a0 2024-05-03T18:40:58.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:40:58.664+0000 7f3d6a59c700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3d6c1047a0 0x7f3d6c102750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:40:58.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:40:58.664+0000 7f3d6a59c700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3d6c1047a0 0x7f3d6c102750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:45448/0 (socket says 172.21.15.113:45448) 2024-05-03T18:40:58.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:40:58.664+0000 7f3d6a59c700 1 -- 172.21.15.113:0/3979740397 learned_addr learned my addr 172.21.15.113:0/3979740397 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-05-03T18:40:58.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:40:58.664+0000 7f3d6b59e700 1 --2- 172.21.15.113:0/3979740397 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d6c105140 0x7f3d6c102c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:40:58.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:40:58.664+0000 7f3d6ad9d700 1 --2- 172.21.15.113:0/3979740397 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3d6c1017c0 0x7f3d6c1040b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:40:58.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:40:58.664+0000 7f3d6a59c700 1 -- 172.21.15.113:0/3979740397 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d6c105140 msgr2=0x7f3d6c102c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:40:58.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:40:58.664+0000 7f3d6a59c700 1 --2- 172.21.15.113:0/3979740397 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d6c105140 0x7f3d6c102c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:58.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:40:58.664+0000 7f3d6a59c700 1 -- 172.21.15.113:0/3979740397 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3d6c1017c0 msgr2=0x7f3d6c1040b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:40:58.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:40:58.664+0000 7f3d6a59c700 1 --2- 172.21.15.113:0/3979740397 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3d6c1017c0 0x7f3d6c1040b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:40:58.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:40:58.664+0000 7f3d6a59c700 1 -- 172.21.15.113:0/3979740397 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d6c103560 con 0x7f3d6c1047a0 2024-05-03T18:40:58.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:40:58.664+0000 7f3d6a59c700 1 --2- 172.21.15.113:0/3979740397 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3d6c1047a0 0x7f3d6c102750 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f3d6000c920 tx=0x7f3d6000cc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:40:58.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:40:58.664+0000 7f3d53fff700 1 -- 172.21.15.113:0/3979740397 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d60007b00 con 0x7f3d6c1047a0 2024-05-03T18:40:58.667 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:40:58.664+0000 7f3d711dd700 1 -- 172.21.15.113:0/3979740397 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d6c1a3820 con 0x7f3d6c1047a0 2024-05-03T18:40:58.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:40:58.664+0000 7f3d53fff700 1 -- 172.21.15.113:0/3979740397 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d60004ce0 con 0x7f3d6c1047a0 2024-05-03T18:40:58.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:40:58.664+0000 7f3d53fff700 1 -- 172.21.15.113:0/3979740397 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d60005500 con 0x7f3d6c1047a0 2024-05-03T18:40:58.668 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:40:58.664+0000 7f3d711dd700 1 -- 172.21.15.113:0/3979740397 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d6c1a3d30 con 0x7f3d6c1047a0 2024-05-03T18:40:58.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:40:58.665+0000 7f3d711dd700 1 -- 172.21.15.113:0/3979740397 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d6c06a190 con 0x7f3d6c1047a0 2024-05-03T18:40:58.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:40:58.667+0000 7f3d53fff700 1 -- 172.21.15.113:0/3979740397 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f3d60007ca0 con 0x7f3d6c1047a0 2024-05-03T18:40:58.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:40:58.668+0000 7f3d53fff700 1 --2- 172.21.15.113:0/3979740397 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f3d540b2880 0x7f3d540b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:40:58.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:40:58.668+0000 7f3d53fff700 1 -- 172.21.15.113:0/3979740397 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4101+0+0 (secure 0 0 0) 0x7f3d600e1190 con 0x7f3d6c1047a0 2024-05-03T18:40:58.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:40:58.668+0000 7f3d6ad9d700 1 --2- 172.21.15.113:0/3979740397 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f3d540b2880 0x7f3d540b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:40:58.669 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:40:58.669+0000 7f3d6ad9d700 1 --2- 172.21.15.113:0/3979740397 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f3d540b2880 0x7f3d540b4d40 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3d5c0038a0 tx=0x7f3d5c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:40:58.672 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:40:58.672+0000 7f3d53fff700 1 -- 172.21.15.113:0/3979740397 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f3d60014030 con 0x7f3d6c1047a0 2024-05-03T18:40:58.793 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:58 smithi045 ceph-mon[33299]: osdmap e32: 5 total, 5 up, 5 in 2024-05-03T18:40:58.793 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:58 smithi045 ceph-mon[33299]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-03T18:40:58.885 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:40:58.884+0000 7f3d711dd700 1 -- 172.21.15.113:0/3979740397 --> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi113:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f3d6c0638e0 con 0x7f3d540b2880 2024-05-03T18:40:58.885 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:58 smithi113 ceph-mon[33540]: osdmap e32: 5 total, 5 up, 5 in 2024-05-03T18:40:58.885 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:58 smithi113 ceph-mon[33540]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-03T18:40:59.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:58 smithi032 ceph-mon[29763]: osdmap e32: 5 total, 5 up, 5 in 2024-05-03T18:40:59.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:58 smithi032 ceph-mon[29763]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-03T18:40:59.764 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:59 smithi113 ceph-mon[33540]: from='client.24244 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi113:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:40:59.764 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:59 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T18:40:59.764 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:59 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T18:40:59.764 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:40:59 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:40:59.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:59 smithi045 ceph-mon[33299]: from='client.24244 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi113:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:40:59.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:59 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T18:40:59.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:59 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T18:40:59.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:40:59 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:41:00.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:59 smithi032 ceph-mon[29763]: from='client.24244 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi113:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:41:00.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:59 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T18:41:00.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:59 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T18:41:00.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:40:59 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:41:00.997 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:41:00 smithi045 ceph-mon[33299]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-03T18:41:01.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:41:00 smithi113 ceph-mon[33540]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-03T18:41:01.067 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:41:00 smithi032 ceph-mon[29763]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-03T18:41:01.997 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:41:01 smithi045 ceph-mon[33299]: from='client.? 172.21.15.113:0/2702106750' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a6688ade-47d7-4a52-807f-3c880a8ffcbb"}]: dispatch 2024-05-03T18:41:01.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:41:01 smithi045 ceph-mon[33299]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a6688ade-47d7-4a52-807f-3c880a8ffcbb"}]: dispatch 2024-05-03T18:41:01.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:41:01 smithi045 ceph-mon[33299]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a6688ade-47d7-4a52-807f-3c880a8ffcbb"}]': finished 2024-05-03T18:41:01.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:41:01 smithi045 ceph-mon[33299]: osdmap e33: 6 total, 5 up, 6 in 2024-05-03T18:41:01.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:41:01 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-03T18:41:02.050 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:41:01 smithi113 ceph-mon[33540]: from='client.? 172.21.15.113:0/2702106750' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a6688ade-47d7-4a52-807f-3c880a8ffcbb"}]: dispatch 2024-05-03T18:41:02.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:41:01 smithi113 ceph-mon[33540]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a6688ade-47d7-4a52-807f-3c880a8ffcbb"}]: dispatch 2024-05-03T18:41:02.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:41:01 smithi113 ceph-mon[33540]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a6688ade-47d7-4a52-807f-3c880a8ffcbb"}]': finished 2024-05-03T18:41:02.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:41:01 smithi113 ceph-mon[33540]: osdmap e33: 6 total, 5 up, 6 in 2024-05-03T18:41:02.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:41:01 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-03T18:41:02.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:41:01 smithi032 ceph-mon[29763]: from='client.? 172.21.15.113:0/2702106750' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a6688ade-47d7-4a52-807f-3c880a8ffcbb"}]: dispatch 2024-05-03T18:41:02.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:41:01 smithi032 ceph-mon[29763]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a6688ade-47d7-4a52-807f-3c880a8ffcbb"}]: dispatch 2024-05-03T18:41:02.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:41:01 smithi032 ceph-mon[29763]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a6688ade-47d7-4a52-807f-3c880a8ffcbb"}]': finished 2024-05-03T18:41:02.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:41:01 smithi032 ceph-mon[29763]: osdmap e33: 6 total, 5 up, 6 in 2024-05-03T18:41:02.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:41:01 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-03T18:41:02.997 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:41:02 smithi045 ceph-mon[33299]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-03T18:41:02.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:41:02 smithi045 ceph-mon[33299]: from='client.? 172.21.15.113:0/3624822116' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T18:41:03.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:41:02 smithi113 ceph-mon[33540]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-03T18:41:03.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:41:02 smithi113 ceph-mon[33540]: from='client.? 172.21.15.113:0/3624822116' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T18:41:03.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:41:02 smithi032 ceph-mon[29763]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-03T18:41:03.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:41:02 smithi032 ceph-mon[29763]: from='client.? 172.21.15.113:0/3624822116' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T18:41:05.247 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:41:04 smithi045 ceph-mon[33299]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-03T18:41:05.300 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:41:04 smithi113 ceph-mon[33540]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-03T18:41:05.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:41:04 smithi032 ceph-mon[29763]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-03T18:41:07.247 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:41:06 smithi045 ceph-mon[33299]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-03T18:41:07.300 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:41:06 smithi113 ceph-mon[33540]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-03T18:41:07.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:41:06 smithi032 ceph-mon[29763]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-03T18:41:09.247 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:41:08 smithi045 ceph-mon[33299]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-03T18:41:09.248 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:41:08 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-03T18:41:09.248 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:41:08 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:41:09.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:41:08 smithi113 ceph-mon[33540]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-03T18:41:09.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:41:08 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-03T18:41:09.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:41:08 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:41:09.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:41:08 smithi032 ceph-mon[29763]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-03T18:41:09.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:41:08 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-03T18:41:09.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:41:08 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:41:10.248 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:41:09 smithi045 ceph-mon[33299]: Deploying daemon osd.5 on smithi113 2024-05-03T18:41:10.300 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:41:09 smithi113 ceph-mon[33540]: Deploying daemon osd.5 on smithi113 2024-05-03T18:41:10.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:41:09 smithi032 ceph-mon[29763]: Deploying daemon osd.5 on smithi113 2024-05-03T18:41:11.247 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:41:10 smithi045 ceph-mon[33299]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-03T18:41:11.261 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:41:10 smithi113 ceph-mon[33540]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-03T18:41:11.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:41:10 smithi032 ceph-mon[29763]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-03T18:41:13.247 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:41:12 smithi045 ceph-mon[33299]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-03T18:41:13.295 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:41:12 smithi113 ceph-mon[33540]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-03T18:41:13.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:41:12 smithi032 ceph-mon[29763]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-03T18:41:15.196 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:41:14 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:41:15.196 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:41:14 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:41:15.196 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:41:14 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:41:15.196 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:41:14 smithi113 ceph-mon[33540]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-03T18:41:15.247 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:41:14 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:41:15.247 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:41:14 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:41:15.247 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:41:14 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:41:15.248 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:41:14 smithi045 ceph-mon[33299]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-03T18:41:15.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:41:14 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:41:15.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:41:14 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:41:15.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:41:14 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:41:15.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:41:14 smithi032 ceph-mon[29763]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-03T18:41:17.247 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:41:16 smithi045 ceph-mon[33299]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-03T18:41:17.247 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:41:16 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:41:17.248 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:41:16 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:41:17.248 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:41:16 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:41:17.248 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:41:16 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:41:17.248 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:41:16 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:41:17.298 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:41:16 smithi113 ceph-mon[33540]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-03T18:41:17.298 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:41:16 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:41:17.298 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:41:16 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:41:17.298 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:41:16 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:41:17.299 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:41:16 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:41:17.299 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:41:16 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:41:17.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:41:16 smithi032 ceph-mon[29763]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-03T18:41:17.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:41:16 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:41:17.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:41:16 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:41:17.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:41:16 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:41:17.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:41:16 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:41:17.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:41:16 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:41:17.655 INFO:teuthology.orchestra.run.smithi113.stdout:Created osd(s) 5 on host 'smithi113' 2024-05-03T18:41:17.655 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:41:17.654+0000 7f3d53fff700 1 -- 172.21.15.113:0/3979740397 <== mgr.14152 v2:172.21.15.32:6800/1796547904 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f3d6c0638e0 con 0x7f3d540b2880 2024-05-03T18:41:17.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:41:17.656+0000 7f3d711dd700 1 -- 172.21.15.113:0/3979740397 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f3d540b2880 msgr2=0x7f3d540b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:41:17.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:41:17.656+0000 7f3d711dd700 1 --2- 172.21.15.113:0/3979740397 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f3d540b2880 0x7f3d540b4d40 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3d5c0038a0 tx=0x7f3d5c005fb0 comp rx=0 tx=0).stop 2024-05-03T18:41:17.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:41:17.656+0000 7f3d711dd700 1 -- 172.21.15.113:0/3979740397 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3d6c1047a0 msgr2=0x7f3d6c102750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:41:17.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:41:17.656+0000 7f3d711dd700 1 --2- 172.21.15.113:0/3979740397 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3d6c1047a0 0x7f3d6c102750 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f3d6000c920 tx=0x7f3d6000cc30 comp rx=0 tx=0).stop 2024-05-03T18:41:17.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:41:17.656+0000 7f3d711dd700 1 -- 172.21.15.113:0/3979740397 shutdown_connections 2024-05-03T18:41:17.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:41:17.656+0000 7f3d711dd700 1 --2- 172.21.15.113:0/3979740397 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f3d540b2880 0x7f3d540b4d40 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:17.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:41:17.656+0000 7f3d711dd700 1 --2- 172.21.15.113:0/3979740397 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d6c105140 0x7f3d6c102c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:17.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:41:17.656+0000 7f3d711dd700 1 --2- 172.21.15.113:0/3979740397 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3d6c1017c0 0x7f3d6c1040b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:17.657 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:41:17.656+0000 7f3d711dd700 1 --2- 172.21.15.113:0/3979740397 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3d6c1047a0 0x7f3d6c102750 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:17.658 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:41:17.656+0000 7f3d711dd700 1 -- 172.21.15.113:0/3979740397 >> 172.21.15.113:0/3979740397 conn(0x7f3d6c0fd100 msgr2=0x7f3d6c107bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:41:17.658 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:41:17.656+0000 7f3d711dd700 1 -- 172.21.15.113:0/3979740397 shutdown_connections 2024-05-03T18:41:17.658 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:41:17.656+0000 7f3d711dd700 1 -- 172.21.15.113:0/3979740397 wait complete. 2024-05-03T18:41:18.257 DEBUG:teuthology.orchestra.run.smithi113:osd.5> sudo journalctl -f -n 0 -u ceph-f500455e-097b-11ef-bc97-c7b262605968@osd.5.service 2024-05-03T18:41:18.259 INFO:tasks.cephadm:Deploying osd.6 on smithi032 with /dev/vg_nvme/lv_2... 2024-05-03T18:41:18.259 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f500455e-097b-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-03T18:41:18.515 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:41:18.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:41:18 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:41:18.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:41:18 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:41:18.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:41:18 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:41:18.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:41:18 smithi113 ceph-mon[33540]: from='osd.5 [v2:172.21.15.113:6808/4085018597,v1:172.21.15.113:6809/4085018597]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-03T18:41:18.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:41:18 smithi113 ceph-mon[33540]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-03T18:41:18.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:41:18 smithi113 ceph-mon[33540]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-03T18:41:18.923 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:41:18 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:41:18.924 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:41:18 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:41:18.924 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:41:18 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:41:18.924 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:41:18 smithi032 ceph-mon[29763]: from='osd.5 [v2:172.21.15.113:6808/4085018597,v1:172.21.15.113:6809/4085018597]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-03T18:41:18.924 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:41:18 smithi032 ceph-mon[29763]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-03T18:41:18.924 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:41:18 smithi032 ceph-mon[29763]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-03T18:41:18.997 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:41:18 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:41:18.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:41:18 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:41:18.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:41:18 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:41:18.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:41:18 smithi045 ceph-mon[33299]: from='osd.5 [v2:172.21.15.113:6808/4085018597,v1:172.21.15.113:6809/4085018597]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-03T18:41:18.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:41:18 smithi045 ceph-mon[33299]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-03T18:41:18.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:41:18 smithi045 ceph-mon[33299]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-03T18:41:19.799 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:41:19 smithi032 ceph-mon[29763]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-03T18:41:19.799 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:41:19 smithi032 ceph-mon[29763]: osdmap e34: 6 total, 5 up, 6 in 2024-05-03T18:41:19.799 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:41:19 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-03T18:41:19.799 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:41:19 smithi032 ceph-mon[29763]: from='osd.5 [v2:172.21.15.113:6808/4085018597,v1:172.21.15.113:6809/4085018597]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi113", "root=default"]}]: dispatch 2024-05-03T18:41:19.799 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:41:19 smithi032 ceph-mon[29763]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi113", "root=default"]}]: dispatch 2024-05-03T18:41:19.997 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:41:19 smithi045 ceph-mon[33299]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-03T18:41:19.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:41:19 smithi045 ceph-mon[33299]: osdmap e34: 6 total, 5 up, 6 in 2024-05-03T18:41:19.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:41:19 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-03T18:41:19.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:41:19 smithi045 ceph-mon[33299]: from='osd.5 [v2:172.21.15.113:6808/4085018597,v1:172.21.15.113:6809/4085018597]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi113", "root=default"]}]: dispatch 2024-05-03T18:41:19.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:41:19 smithi045 ceph-mon[33299]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi113", "root=default"]}]: dispatch 2024-05-03T18:41:20.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:41:19 smithi113 ceph-mon[33540]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-03T18:41:20.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:41:19 smithi113 ceph-mon[33540]: osdmap e34: 6 total, 5 up, 6 in 2024-05-03T18:41:20.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:41:19 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-03T18:41:20.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:41:19 smithi113 ceph-mon[33540]: from='osd.5 [v2:172.21.15.113:6808/4085018597,v1:172.21.15.113:6809/4085018597]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi113", "root=default"]}]: dispatch 2024-05-03T18:41:20.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:41:19 smithi113 ceph-mon[33540]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi113", "root=default"]}]: dispatch 2024-05-03T18:41:20.354 INFO:teuthology.orchestra.run.smithi032.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-03T18:41:20.355 INFO:teuthology.orchestra.run.smithi032.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-03T18:41:20.355 INFO:teuthology.orchestra.run.smithi032.stderr: stderr: 10+0 records in 2024-05-03T18:41:20.355 INFO:teuthology.orchestra.run.smithi032.stderr:10+0 records out 2024-05-03T18:41:20.355 INFO:teuthology.orchestra.run.smithi032.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00866354 s, 1.2 GB/s 2024-05-03T18:41:20.355 INFO:teuthology.orchestra.run.smithi032.stderr:--> Zapping successful for: 2024-05-03T18:41:20.622 INFO:journalctl@ceph.osd.5.smithi113.stdout:May 03 18:41:20 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-5[45660]: 2024-05-03T18:41:20.364+0000 7f0c68cb7700 -1 osd.5 0 waiting for initial osdmap 2024-05-03T18:41:20.623 INFO:journalctl@ceph.osd.5.smithi113.stdout:May 03 18:41:20 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-5[45660]: 2024-05-03T18:41:20.491+0000 7f0c60aa2700 -1 osd.5 35 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-03T18:41:20.975 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:41:20 smithi113 ceph-mon[33540]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi113", "root=default"]}]': finished 2024-05-03T18:41:20.975 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:41:20 smithi113 ceph-mon[33540]: osdmap e35: 6 total, 5 up, 6 in 2024-05-03T18:41:20.975 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:41:20 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-03T18:41:20.975 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:41:20 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-03T18:41:20.975 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:41:20 smithi113 ceph-mon[33540]: pgmap v132: 1 pgs: 1 peering; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-03T18:41:20.975 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:41:20 smithi113 ceph-mon[33540]: from='osd.5 ' entity='osd.5' 2024-05-03T18:41:20.997 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:41:20 smithi045 ceph-mon[33299]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi113", "root=default"]}]': finished 2024-05-03T18:41:20.997 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:41:20 smithi045 ceph-mon[33299]: osdmap e35: 6 total, 5 up, 6 in 2024-05-03T18:41:20.997 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:41:20 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-03T18:41:20.997 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:41:20 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-03T18:41:20.997 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:41:20 smithi045 ceph-mon[33299]: pgmap v132: 1 pgs: 1 peering; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-03T18:41:20.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:41:20 smithi045 ceph-mon[33299]: from='osd.5 ' entity='osd.5' 2024-05-03T18:41:21.032 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:41:20 smithi032 ceph-mon[29763]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi113", "root=default"]}]': finished 2024-05-03T18:41:21.032 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:41:20 smithi032 ceph-mon[29763]: osdmap e35: 6 total, 5 up, 6 in 2024-05-03T18:41:21.032 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:41:20 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-03T18:41:21.032 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:41:20 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-03T18:41:21.032 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:41:20 smithi032 ceph-mon[29763]: pgmap v132: 1 pgs: 1 peering; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-05-03T18:41:21.032 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:41:20 smithi032 ceph-mon[29763]: from='osd.5 ' entity='osd.5' 2024-05-03T18:41:21.066 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f500455e-097b-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi032:vg_nvme/lv_2 2024-05-03T18:41:21.316 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:41:21.735 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:41:21 smithi032 ceph-mon[29763]: purged_snaps scrub starts 2024-05-03T18:41:21.735 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:41:21 smithi032 ceph-mon[29763]: purged_snaps scrub ok 2024-05-03T18:41:21.736 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:41:21 smithi032 ceph-mon[29763]: osd.5 [v2:172.21.15.113:6808/4085018597,v1:172.21.15.113:6809/4085018597] boot 2024-05-03T18:41:21.736 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:41:21 smithi032 ceph-mon[29763]: osdmap e36: 6 total, 6 up, 6 in 2024-05-03T18:41:21.736 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:41:21 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-03T18:41:21.736 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:41:21 smithi032 ceph-mon[29763]: Detected new or changed devices on smithi113 2024-05-03T18:41:21.736 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:41:21 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:41:21.736 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:41:21 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:41:21.736 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:41:21 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:41:21.736 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:41:21 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:41:21.736 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:41:21 smithi032 ceph-mon[29763]: Adjusting osd_memory_target on smithi113 to 8544M 2024-05-03T18:41:21.736 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:41:21 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:41:21.736 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:41:21 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:41:21.736 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:41:21 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:41:21.736 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:41:21 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:41:21.736 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:41:21 smithi032 ceph-mon[29763]: osdmap e37: 6 total, 6 up, 6 in 2024-05-03T18:41:21.997 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:41:21 smithi045 ceph-mon[33299]: purged_snaps scrub starts 2024-05-03T18:41:21.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:41:21 smithi045 ceph-mon[33299]: purged_snaps scrub ok 2024-05-03T18:41:21.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:41:21 smithi045 ceph-mon[33299]: osd.5 [v2:172.21.15.113:6808/4085018597,v1:172.21.15.113:6809/4085018597] boot 2024-05-03T18:41:21.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:41:21 smithi045 ceph-mon[33299]: osdmap e36: 6 total, 6 up, 6 in 2024-05-03T18:41:21.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:41:21 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-03T18:41:21.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:41:21 smithi045 ceph-mon[33299]: Detected new or changed devices on smithi113 2024-05-03T18:41:21.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:41:21 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:41:21.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:41:21 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:41:21.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:41:21 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:41:21.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:41:21 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:41:21.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:41:21 smithi045 ceph-mon[33299]: Adjusting osd_memory_target on smithi113 to 8544M 2024-05-03T18:41:21.999 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:41:21 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:41:21.999 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:41:21 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:41:21.999 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:41:21 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:41:21.999 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:41:21 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:41:21.999 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:41:21 smithi045 ceph-mon[33299]: osdmap e37: 6 total, 6 up, 6 in 2024-05-03T18:41:22.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:41:21 smithi113 ceph-mon[33540]: purged_snaps scrub starts 2024-05-03T18:41:22.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:41:21 smithi113 ceph-mon[33540]: purged_snaps scrub ok 2024-05-03T18:41:22.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:41:21 smithi113 ceph-mon[33540]: osd.5 [v2:172.21.15.113:6808/4085018597,v1:172.21.15.113:6809/4085018597] boot 2024-05-03T18:41:22.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:41:21 smithi113 ceph-mon[33540]: osdmap e36: 6 total, 6 up, 6 in 2024-05-03T18:41:22.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:41:21 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-03T18:41:22.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:41:21 smithi113 ceph-mon[33540]: Detected new or changed devices on smithi113 2024-05-03T18:41:22.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:41:21 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:41:22.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:41:21 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:41:22.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:41:21 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:41:22.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:41:21 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:41:22.052 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:41:21 smithi113 ceph-mon[33540]: Adjusting osd_memory_target on smithi113 to 8544M 2024-05-03T18:41:22.052 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:41:21 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:41:22.052 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:41:21 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:41:22.052 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:41:21 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:41:22.052 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:41:21 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:41:22.052 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:41:21 smithi113 ceph-mon[33540]: osdmap e37: 6 total, 6 up, 6 in 2024-05-03T18:41:22.844 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:41:22.841+0000 7fdbfe9b4700 1 -- 172.21.15.32:0/547199783 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdbf8103500 msgr2=0x7fdbf8103920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:41:22.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:41:22.841+0000 7fdbfe9b4700 1 --2- 172.21.15.32:0/547199783 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdbf8103500 0x7fdbf8103920 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fdbe80099c0 tx=0x7fdbe8009cd0 comp rx=0 tx=0).stop 2024-05-03T18:41:22.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:41:22.841+0000 7fdbfe9b4700 1 -- 172.21.15.32:0/547199783 shutdown_connections 2024-05-03T18:41:22.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:41:22.841+0000 7fdbfe9b4700 1 --2- 172.21.15.32:0/547199783 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdbf81050a0 0x7fdbf810b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:22.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:41:22.842+0000 7fdbfe9b4700 1 --2- 172.21.15.32:0/547199783 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdbf8103500 0x7fdbf8103920 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:22.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:41:22.842+0000 7fdbfe9b4700 1 --2- 172.21.15.32:0/547199783 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdbf8104700 0x7fdbf8104b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:22.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:41:22.842+0000 7fdbfe9b4700 1 -- 172.21.15.32:0/547199783 >> 172.21.15.32:0/547199783 conn(0x7fdbf80feaa0 msgr2=0x7fdbf8100ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:41:22.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:41:22.842+0000 7fdbfe9b4700 1 -- 172.21.15.32:0/547199783 shutdown_connections 2024-05-03T18:41:22.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:41:22.842+0000 7fdbfe9b4700 1 -- 172.21.15.32:0/547199783 wait complete. 2024-05-03T18:41:22.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:41:22.842+0000 7fdbfe9b4700 1 Processor -- start 2024-05-03T18:41:22.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:41:22.842+0000 7fdbfe9b4700 1 -- start start 2024-05-03T18:41:22.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:41:22.843+0000 7fdbfe9b4700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdbf8103500 0x7fdbf8199740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:41:22.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:41:22.843+0000 7fdbfe9b4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdbf8104700 0x7fdbf8199c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:41:22.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:41:22.843+0000 7fdbfe9b4700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdbf81050a0 0x7fdbf81a0d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:41:22.845 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:41:22.843+0000 7fdbfe9b4700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fdbf810f070 con 0x7fdbf8104700 2024-05-03T18:41:22.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:41:22.843+0000 7fdbfe9b4700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fdbf810eef0 con 0x7fdbf8103500 2024-05-03T18:41:22.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:41:22.843+0000 7fdbfe9b4700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fdbf810f1f0 con 0x7fdbf81050a0 2024-05-03T18:41:22.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:41:22.843+0000 7fdbf77fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdbf8104700 0x7fdbf8199c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:41:22.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:41:22.843+0000 7fdbf77fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdbf8104700 0x7fdbf8199c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50512/0 (socket says 172.21.15.32:50512) 2024-05-03T18:41:22.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:41:22.843+0000 7fdbf77fe700 1 -- 172.21.15.32:0/1632716263 learned_addr learned my addr 172.21.15.32:0/1632716263 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:41:22.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:41:22.843+0000 7fdbf7fff700 1 --2- 172.21.15.32:0/1632716263 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdbf8103500 0x7fdbf8199740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:41:22.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:41:22.843+0000 7fdbfcf51700 1 --2- 172.21.15.32:0/1632716263 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdbf81050a0 0x7fdbf81a0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:41:22.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:41:22.843+0000 7fdbf77fe700 1 -- 172.21.15.32:0/1632716263 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdbf81050a0 msgr2=0x7fdbf81a0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:41:22.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:41:22.843+0000 7fdbf77fe700 1 --2- 172.21.15.32:0/1632716263 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdbf81050a0 0x7fdbf81a0d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:22.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:41:22.843+0000 7fdbf77fe700 1 -- 172.21.15.32:0/1632716263 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdbf8103500 msgr2=0x7fdbf8199740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:41:22.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:41:22.843+0000 7fdbf77fe700 1 --2- 172.21.15.32:0/1632716263 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdbf8103500 0x7fdbf8199740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:22.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:41:22.843+0000 7fdbf77fe700 1 -- 172.21.15.32:0/1632716263 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbf81a13a0 con 0x7fdbf8104700 2024-05-03T18:41:22.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:41:22.843+0000 7fdbf7fff700 1 --2- 172.21.15.32:0/1632716263 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdbf8103500 0x7fdbf8199740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:41:22.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:41:22.843+0000 7fdbfcf51700 1 --2- 172.21.15.32:0/1632716263 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdbf81050a0 0x7fdbf81a0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:41:22.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:41:22.843+0000 7fdbf77fe700 1 --2- 172.21.15.32:0/1632716263 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdbf8104700 0x7fdbf8199c80 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fdbec00b560 tx=0x7fdbec00b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:41:22.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:41:22.843+0000 7fdbf57fa700 1 -- 172.21.15.32:0/1632716263 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbec015020 con 0x7fdbf8104700 2024-05-03T18:41:22.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:41:22.843+0000 7fdbf57fa700 1 -- 172.21.15.32:0/1632716263 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdbec011ce0 con 0x7fdbf8104700 2024-05-03T18:41:22.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:41:22.844+0000 7fdbfe9b4700 1 -- 172.21.15.32:0/1632716263 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbf81a1690 con 0x7fdbf8104700 2024-05-03T18:41:22.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:41:22.844+0000 7fdbf57fa700 1 -- 172.21.15.32:0/1632716263 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbec010da0 con 0x7fdbf8104700 2024-05-03T18:41:22.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:41:22.844+0000 7fdbfe9b4700 1 -- 172.21.15.32:0/1632716263 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbf81a1c50 con 0x7fdbf8104700 2024-05-03T18:41:22.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:41:22.844+0000 7fdbfe9b4700 1 -- 172.21.15.32:0/1632716263 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbf806a190 con 0x7fdbf8104700 2024-05-03T18:41:22.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:41:22.846+0000 7fdbf57fa700 1 -- 172.21.15.32:0/1632716263 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fdbec018020 con 0x7fdbf8104700 2024-05-03T18:41:22.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:41:22.847+0000 7fdbf57fa700 1 --2- 172.21.15.32:0/1632716263 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fdbe00b2900 0x7fdbe00b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:41:22.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:41:22.847+0000 7fdbf7fff700 1 --2- 172.21.15.32:0/1632716263 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fdbe00b2900 0x7fdbe00b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:41:22.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:41:22.847+0000 7fdbf57fa700 1 -- 172.21.15.32:0/1632716263 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4533+0+0 (secure 0 0 0) 0x7fdbec0e10f0 con 0x7fdbf8104700 2024-05-03T18:41:22.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:41:22.847+0000 7fdbf7fff700 1 --2- 172.21.15.32:0/1632716263 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fdbe00b2900 0x7fdbe00b4dc0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fdbe8006010 tx=0x7fdbe8009f10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:41:22.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:41:22.851+0000 7fdbf57fa700 1 -- 172.21.15.32:0/1632716263 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fdbec01f020 con 0x7fdbf8104700 2024-05-03T18:41:22.997 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:41:22 smithi045 ceph-mon[33299]: pgmap v135: 1 pgs: 1 peering; 577 KiB data, 578 MiB used, 536 GiB / 536 GiB avail 2024-05-03T18:41:23.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:41:22 smithi113 ceph-mon[33540]: pgmap v135: 1 pgs: 1 peering; 577 KiB data, 578 MiB used, 536 GiB / 536 GiB avail 2024-05-03T18:41:23.067 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:41:22 smithi032 ceph-mon[29763]: pgmap v135: 1 pgs: 1 peering; 577 KiB data, 578 MiB used, 536 GiB / 536 GiB avail 2024-05-03T18:41:23.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:41:23.072+0000 7fdbfe9b4700 1 -- 172.21.15.32:0/1632716263 --> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi032:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fdbf80638e0 con 0x7fdbe00b2900 2024-05-03T18:41:23.894 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:41:23 smithi032 ceph-mon[29763]: osdmap e38: 6 total, 6 up, 6 in 2024-05-03T18:41:23.894 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:41:23 smithi032 ceph-mon[29763]: from='client.14373 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi032:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:41:23.894 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:41:23 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T18:41:23.894 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:41:23 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T18:41:23.894 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:41:23 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:41:23.997 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:41:23 smithi045 ceph-mon[33299]: osdmap e38: 6 total, 6 up, 6 in 2024-05-03T18:41:23.997 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:41:23 smithi045 ceph-mon[33299]: from='client.14373 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi032:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:41:23.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:41:23 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T18:41:23.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:41:23 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T18:41:23.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:41:23 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:41:24.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:41:23 smithi113 ceph-mon[33540]: osdmap e38: 6 total, 6 up, 6 in 2024-05-03T18:41:24.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:41:23 smithi113 ceph-mon[33540]: from='client.14373 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi032:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:41:24.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:41:23 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T18:41:24.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:41:23 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T18:41:24.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:41:23 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:41:24.906 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:41:24 smithi032 ceph-mon[29763]: pgmap v137: 1 pgs: 1 peering; 577 KiB data, 577 MiB used, 536 GiB / 536 GiB avail 2024-05-03T18:41:24.997 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:41:24 smithi045 ceph-mon[33299]: pgmap v137: 1 pgs: 1 peering; 577 KiB data, 577 MiB used, 536 GiB / 536 GiB avail 2024-05-03T18:41:25.050 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:41:24 smithi113 ceph-mon[33540]: pgmap v137: 1 pgs: 1 peering; 577 KiB data, 577 MiB used, 536 GiB / 536 GiB avail 2024-05-03T18:41:26.092 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:41:25 smithi032 ceph-mon[29763]: from='client.? 172.21.15.32:0/40736757' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1d9ebda9-55c7-473a-9dc0-fc60845bbfe9"}]: dispatch 2024-05-03T18:41:26.092 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:41:25 smithi032 ceph-mon[29763]: from='client.? 172.21.15.32:0/40736757' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1d9ebda9-55c7-473a-9dc0-fc60845bbfe9"}]': finished 2024-05-03T18:41:26.092 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:41:25 smithi032 ceph-mon[29763]: osdmap e39: 7 total, 6 up, 7 in 2024-05-03T18:41:26.092 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:41:25 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-03T18:41:26.223 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:41:25 smithi113 ceph-mon[33540]: from='client.? 172.21.15.32:0/40736757' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1d9ebda9-55c7-473a-9dc0-fc60845bbfe9"}]: dispatch 2024-05-03T18:41:26.223 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:41:25 smithi113 ceph-mon[33540]: from='client.? 172.21.15.32:0/40736757' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1d9ebda9-55c7-473a-9dc0-fc60845bbfe9"}]': finished 2024-05-03T18:41:26.223 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:41:25 smithi113 ceph-mon[33540]: osdmap e39: 7 total, 6 up, 7 in 2024-05-03T18:41:26.223 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:41:25 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-03T18:41:26.247 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:41:25 smithi045 ceph-mon[33299]: from='client.? 172.21.15.32:0/40736757' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1d9ebda9-55c7-473a-9dc0-fc60845bbfe9"}]: dispatch 2024-05-03T18:41:26.248 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:41:25 smithi045 ceph-mon[33299]: from='client.? 172.21.15.32:0/40736757' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1d9ebda9-55c7-473a-9dc0-fc60845bbfe9"}]': finished 2024-05-03T18:41:26.248 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:41:25 smithi045 ceph-mon[33299]: osdmap e39: 7 total, 6 up, 7 in 2024-05-03T18:41:26.248 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:41:25 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-03T18:41:27.143 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:41:26 smithi113 ceph-mon[33540]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail; 109 KiB/s, 0 objects/s recovering 2024-05-03T18:41:27.143 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:41:26 smithi113 ceph-mon[33540]: from='client.? 172.21.15.32:0/1039917419' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T18:41:27.149 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:41:26 smithi045 ceph-mon[33299]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail; 109 KiB/s, 0 objects/s recovering 2024-05-03T18:41:27.149 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:41:26 smithi045 ceph-mon[33299]: from='client.? 172.21.15.32:0/1039917419' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T18:41:27.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:41:26 smithi032 ceph-mon[29763]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail; 109 KiB/s, 0 objects/s recovering 2024-05-03T18:41:27.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:41:26 smithi032 ceph-mon[29763]: from='client.? 172.21.15.32:0/1039917419' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T18:41:29.247 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:41:28 smithi045 ceph-mon[33299]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail; 91 KiB/s, 0 objects/s recovering 2024-05-03T18:41:29.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:41:28 smithi113 ceph-mon[33540]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail; 91 KiB/s, 0 objects/s recovering 2024-05-03T18:41:29.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:41:28 smithi032 ceph-mon[29763]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail; 91 KiB/s, 0 objects/s recovering 2024-05-03T18:41:31.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:41:30 smithi032 ceph-mon[29763]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-03T18:41:31.247 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:41:30 smithi045 ceph-mon[33299]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-03T18:41:31.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:41:30 smithi113 ceph-mon[33540]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-03T18:41:33.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:41:32 smithi032 ceph-mon[29763]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-05-03T18:41:33.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:41:32 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-03T18:41:33.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:41:32 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:41:33.247 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:41:32 smithi045 ceph-mon[33299]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-05-03T18:41:33.247 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:41:32 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-03T18:41:33.248 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:41:32 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:41:33.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:41:32 smithi113 ceph-mon[33540]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-05-03T18:41:33.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:41:32 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-03T18:41:33.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:41:32 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:41:34.151 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:41:33 smithi032 ceph-mon[29763]: Deploying daemon osd.6 on smithi032 2024-05-03T18:41:34.247 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:41:33 smithi045 ceph-mon[33299]: Deploying daemon osd.6 on smithi032 2024-05-03T18:41:34.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:41:33 smithi113 ceph-mon[33540]: Deploying daemon osd.6 on smithi032 2024-05-03T18:41:35.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:41:34 smithi032 ceph-mon[29763]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-03T18:41:35.247 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:41:34 smithi045 ceph-mon[33299]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-03T18:41:35.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:41:34 smithi113 ceph-mon[33540]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-03T18:41:37.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:41:36 smithi032 ceph-mon[29763]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-05-03T18:41:37.247 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:41:36 smithi045 ceph-mon[33299]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-05-03T18:41:37.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:41:36 smithi113 ceph-mon[33540]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-05-03T18:41:39.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:41:39 smithi113 ceph-mon[33540]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-05-03T18:41:39.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:41:39 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:41:39.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:41:39 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:41:39.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:41:39 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:41:39.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:41:39 smithi032 ceph-mon[29763]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-05-03T18:41:39.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:41:39 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:41:39.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:41:39 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:41:39.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:41:39 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:41:39.997 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:41:39 smithi045 ceph-mon[33299]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-05-03T18:41:39.997 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:41:39 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:41:39.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:41:39 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:41:39.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:41:39 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:41:41.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:41:41 smithi113 ceph-mon[33540]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-05-03T18:41:41.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:41:41 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:41:41.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:41:41 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:41:41.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:41:41 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:41:41.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:41:41 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:41:41.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:41:41 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:41:41.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:41:41 smithi032 ceph-mon[29763]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-05-03T18:41:41.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:41:41 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:41:41.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:41:41 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:41:41.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:41:41 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:41:41.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:41:41 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:41:41.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:41:41 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:41:41.997 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:41:41 smithi045 ceph-mon[33299]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-05-03T18:41:41.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:41:41 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:41:41.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:41:41 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:41:41.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:41:41 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:41:41.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:41:41 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:41:41.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:41:41 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:41:42.378 INFO:teuthology.orchestra.run.smithi032.stdout:Created osd(s) 6 on host 'smithi032' 2024-05-03T18:41:42.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:41:42.377+0000 7fdbf57fa700 1 -- 172.21.15.32:0/1632716263 <== mgr.14152 v2:172.21.15.32:6800/1796547904 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fdbf80638e0 con 0x7fdbe00b2900 2024-05-03T18:41:42.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:41:42.380+0000 7fdbfe9b4700 1 -- 172.21.15.32:0/1632716263 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fdbe00b2900 msgr2=0x7fdbe00b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:41:42.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:41:42.380+0000 7fdbfe9b4700 1 --2- 172.21.15.32:0/1632716263 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fdbe00b2900 0x7fdbe00b4dc0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fdbe8006010 tx=0x7fdbe8009f10 comp rx=0 tx=0).stop 2024-05-03T18:41:42.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:41:42.380+0000 7fdbfe9b4700 1 -- 172.21.15.32:0/1632716263 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdbf8104700 msgr2=0x7fdbf8199c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:41:42.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:41:42.380+0000 7fdbfe9b4700 1 --2- 172.21.15.32:0/1632716263 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdbf8104700 0x7fdbf8199c80 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fdbec00b560 tx=0x7fdbec00b920 comp rx=0 tx=0).stop 2024-05-03T18:41:42.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:41:42.380+0000 7fdbfe9b4700 1 -- 172.21.15.32:0/1632716263 shutdown_connections 2024-05-03T18:41:42.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:41:42.380+0000 7fdbfe9b4700 1 --2- 172.21.15.32:0/1632716263 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fdbe00b2900 0x7fdbe00b4dc0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:42.381 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:41:42.380+0000 7fdbfe9b4700 1 --2- 172.21.15.32:0/1632716263 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdbf81050a0 0x7fdbf81a0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:42.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:41:42.380+0000 7fdbfe9b4700 1 --2- 172.21.15.32:0/1632716263 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdbf8103500 0x7fdbf8199740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:42.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:41:42.380+0000 7fdbfe9b4700 1 --2- 172.21.15.32:0/1632716263 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdbf8104700 0x7fdbf8199c80 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:42.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:41:42.380+0000 7fdbfe9b4700 1 -- 172.21.15.32:0/1632716263 >> 172.21.15.32:0/1632716263 conn(0x7fdbf80feaa0 msgr2=0x7fdbf8107a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:41:42.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:41:42.380+0000 7fdbfe9b4700 1 -- 172.21.15.32:0/1632716263 shutdown_connections 2024-05-03T18:41:42.382 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:41:42.380+0000 7fdbfe9b4700 1 -- 172.21.15.32:0/1632716263 wait complete. 2024-05-03T18:41:42.977 DEBUG:teuthology.orchestra.run.smithi032:osd.6> sudo journalctl -f -n 0 -u ceph-f500455e-097b-11ef-bc97-c7b262605968@osd.6.service 2024-05-03T18:41:42.979 INFO:tasks.cephadm:Deploying osd.7 on smithi045 with /dev/vg_nvme/lv_2... 2024-05-03T18:41:42.979 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f500455e-097b-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-03T18:41:43.236 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/mon.b/config 2024-05-03T18:41:43.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:41:43 smithi032 ceph-mon[29763]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-05-03T18:41:43.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:41:43 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:41:43.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:41:43 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:41:43.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:41:43 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:41:43.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:41:43 smithi032 ceph-mon[29763]: from='osd.6 [v2:172.21.15.32:6818/791407796,v1:172.21.15.32:6819/791407796]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-03T18:41:43.660 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:41:43 smithi045 ceph-mon[33299]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-05-03T18:41:43.660 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:41:43 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:41:43.660 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:41:43 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:41:43.660 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:41:43 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:41:43.660 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:41:43 smithi045 ceph-mon[33299]: from='osd.6 [v2:172.21.15.32:6818/791407796,v1:172.21.15.32:6819/791407796]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-03T18:41:43.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:41:43 smithi113 ceph-mon[33540]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-05-03T18:41:43.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:41:43 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:41:43.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:41:43 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:41:43.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:41:43 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:41:43.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:41:43 smithi113 ceph-mon[33540]: from='osd.6 [v2:172.21.15.32:6818/791407796,v1:172.21.15.32:6819/791407796]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-03T18:41:44.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:41:44 smithi032 ceph-mon[29763]: from='osd.6 [v2:172.21.15.32:6818/791407796,v1:172.21.15.32:6819/791407796]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-03T18:41:44.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:41:44 smithi032 ceph-mon[29763]: osdmap e40: 7 total, 6 up, 7 in 2024-05-03T18:41:44.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:41:44 smithi032 ceph-mon[29763]: from='osd.6 [v2:172.21.15.32:6818/791407796,v1:172.21.15.32:6819/791407796]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi032", "root=default"]}]: dispatch 2024-05-03T18:41:44.358 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:41:44 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-03T18:41:44.610 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:41:44 smithi045 ceph-mon[33299]: from='osd.6 [v2:172.21.15.32:6818/791407796,v1:172.21.15.32:6819/791407796]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-03T18:41:44.611 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:41:44 smithi045 ceph-mon[33299]: osdmap e40: 7 total, 6 up, 7 in 2024-05-03T18:41:44.611 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:41:44 smithi045 ceph-mon[33299]: from='osd.6 [v2:172.21.15.32:6818/791407796,v1:172.21.15.32:6819/791407796]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi032", "root=default"]}]: dispatch 2024-05-03T18:41:44.611 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:41:44 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-03T18:41:44.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:41:44 smithi113 ceph-mon[33540]: from='osd.6 [v2:172.21.15.32:6818/791407796,v1:172.21.15.32:6819/791407796]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-03T18:41:44.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:41:44 smithi113 ceph-mon[33540]: osdmap e40: 7 total, 6 up, 7 in 2024-05-03T18:41:44.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:41:44 smithi113 ceph-mon[33540]: from='osd.6 [v2:172.21.15.32:6818/791407796,v1:172.21.15.32:6819/791407796]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi032", "root=default"]}]: dispatch 2024-05-03T18:41:44.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:41:44 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-03T18:41:44.818 INFO:journalctl@ceph.osd.6.smithi032.stdout:May 03 18:41:44 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-6[55806]: 2024-05-03T18:41:44.356+0000 7f7d9e10b700 -1 osd.6 0 waiting for initial osdmap 2024-05-03T18:41:44.818 INFO:journalctl@ceph.osd.6.smithi032.stdout:May 03 18:41:44 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-6[55806]: 2024-05-03T18:41:44.366+0000 7f7d99efe700 -1 osd.6 41 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-03T18:41:45.341 INFO:teuthology.orchestra.run.smithi045.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-03T18:41:45.341 INFO:teuthology.orchestra.run.smithi045.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-03T18:41:45.341 INFO:teuthology.orchestra.run.smithi045.stderr: stderr: 10+0 records in 2024-05-03T18:41:45.341 INFO:teuthology.orchestra.run.smithi045.stderr:10+0 records out 2024-05-03T18:41:45.341 INFO:teuthology.orchestra.run.smithi045.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0084636 s, 1.2 GB/s 2024-05-03T18:41:45.341 INFO:teuthology.orchestra.run.smithi045.stderr:--> Zapping successful for: 2024-05-03T18:41:45.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:41:45 smithi032 ceph-mon[29763]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-05-03T18:41:45.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:41:45 smithi032 ceph-mon[29763]: from='osd.6 [v2:172.21.15.32:6818/791407796,v1:172.21.15.32:6819/791407796]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi032", "root=default"]}]': finished 2024-05-03T18:41:45.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:41:45 smithi032 ceph-mon[29763]: osdmap e41: 7 total, 6 up, 7 in 2024-05-03T18:41:45.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:41:45 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-03T18:41:45.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:41:45 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-03T18:41:45.654 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:41:45 smithi045 ceph-mon[33299]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-05-03T18:41:45.654 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:41:45 smithi045 ceph-mon[33299]: from='osd.6 [v2:172.21.15.32:6818/791407796,v1:172.21.15.32:6819/791407796]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi032", "root=default"]}]': finished 2024-05-03T18:41:45.654 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:41:45 smithi045 ceph-mon[33299]: osdmap e41: 7 total, 6 up, 7 in 2024-05-03T18:41:45.654 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:41:45 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-03T18:41:45.654 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:41:45 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-03T18:41:45.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:41:45 smithi113 ceph-mon[33540]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-05-03T18:41:45.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:41:45 smithi113 ceph-mon[33540]: from='osd.6 [v2:172.21.15.32:6818/791407796,v1:172.21.15.32:6819/791407796]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi032", "root=default"]}]': finished 2024-05-03T18:41:45.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:41:45 smithi113 ceph-mon[33540]: osdmap e41: 7 total, 6 up, 7 in 2024-05-03T18:41:45.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:41:45 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-03T18:41:45.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:41:45 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-03T18:41:45.925 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f500455e-097b-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi045:vg_nvme/lv_2 2024-05-03T18:41:46.191 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/mon.b/config 2024-05-03T18:41:46.631 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:41:46 smithi045 ceph-mon[33299]: purged_snaps scrub starts 2024-05-03T18:41:46.632 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:41:46 smithi045 ceph-mon[33299]: purged_snaps scrub ok 2024-05-03T18:41:46.632 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:41:46 smithi045 ceph-mon[33299]: osd.6 [v2:172.21.15.32:6818/791407796,v1:172.21.15.32:6819/791407796] boot 2024-05-03T18:41:46.632 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:41:46 smithi045 ceph-mon[33299]: osdmap e42: 7 total, 7 up, 7 in 2024-05-03T18:41:46.632 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:41:46 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-03T18:41:46.632 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:41:46 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:41:46.632 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:41:46 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:41:46.632 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:41:46 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:41:46.632 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:41:46 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:41:46.632 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:41:46 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:41:46.632 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:41:46 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:41:46.632 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:41:46 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:41:46.633 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:41:46 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:41:46.633 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:41:46 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:41:46.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:41:46 smithi113 ceph-mon[33540]: purged_snaps scrub starts 2024-05-03T18:41:46.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:41:46 smithi113 ceph-mon[33540]: purged_snaps scrub ok 2024-05-03T18:41:46.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:41:46 smithi113 ceph-mon[33540]: osd.6 [v2:172.21.15.32:6818/791407796,v1:172.21.15.32:6819/791407796] boot 2024-05-03T18:41:46.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:41:46 smithi113 ceph-mon[33540]: osdmap e42: 7 total, 7 up, 7 in 2024-05-03T18:41:46.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:41:46 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-03T18:41:46.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:41:46 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:41:46.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:41:46 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:41:46.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:41:46 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:41:46.802 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:41:46 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:41:46.802 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:41:46 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:41:46.802 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:41:46 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:41:46.802 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:41:46 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:41:46.802 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:41:46 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:41:46.802 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:41:46 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:41:46.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:41:46 smithi032 ceph-mon[29763]: purged_snaps scrub starts 2024-05-03T18:41:46.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:41:46 smithi032 ceph-mon[29763]: purged_snaps scrub ok 2024-05-03T18:41:46.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:41:46 smithi032 ceph-mon[29763]: osd.6 [v2:172.21.15.32:6818/791407796,v1:172.21.15.32:6819/791407796] boot 2024-05-03T18:41:46.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:41:46 smithi032 ceph-mon[29763]: osdmap e42: 7 total, 7 up, 7 in 2024-05-03T18:41:46.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:41:46 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-03T18:41:46.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:41:46 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:41:46.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:41:46 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:41:46.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:41:46 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:41:46.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:41:46 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:41:46.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:41:46 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:41:46.819 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:41:46 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:41:46.819 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:41:46 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:41:46.819 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:41:46 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:41:46.819 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:41:46 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:41:47.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:41:47 smithi045 ceph-mon[33299]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-05-03T18:41:47.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:41:47 smithi045 ceph-mon[33299]: Detected new or changed devices on smithi032 2024-05-03T18:41:47.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:41:47 smithi045 ceph-mon[33299]: Adjusting osd_memory_target on smithi032 to 5697M 2024-05-03T18:41:47.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:41:47 smithi045 ceph-mon[33299]: osdmap e43: 7 total, 7 up, 7 in 2024-05-03T18:41:47.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:41:47.705+0000 7f6416052700 1 -- 172.21.15.45:0/509073000 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6410103fb0 msgr2=0x7f64101063a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:41:47.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:41:47.705+0000 7f6416052700 1 --2- 172.21.15.45:0/509073000 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6410103fb0 0x7f64101063a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f6400009990 tx=0x7f6400009ca0 comp rx=0 tx=0).stop 2024-05-03T18:41:47.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:41:47.705+0000 7f6416052700 1 -- 172.21.15.45:0/509073000 shutdown_connections 2024-05-03T18:41:47.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:41:47.705+0000 7f6416052700 1 --2- 172.21.15.45:0/509073000 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6410109210 0x7f641010b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:47.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:41:47.705+0000 7f6416052700 1 --2- 172.21.15.45:0/509073000 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6410103fb0 0x7f64101063a0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:47.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:41:47.705+0000 7f6416052700 1 --2- 172.21.15.45:0/509073000 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64101068e0 0x7f6410108cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:47.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:41:47.705+0000 7f6416052700 1 -- 172.21.15.45:0/509073000 >> 172.21.15.45:0/509073000 conn(0x7f64100fd9c0 msgr2=0x7f64100ffe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:41:47.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:41:47.705+0000 7f6416052700 1 -- 172.21.15.45:0/509073000 shutdown_connections 2024-05-03T18:41:47.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:41:47.705+0000 7f6416052700 1 -- 172.21.15.45:0/509073000 wait complete. 2024-05-03T18:41:47.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:41:47.705+0000 7f6416052700 1 Processor -- start 2024-05-03T18:41:47.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:41:47.706+0000 7f6416052700 1 -- start start 2024-05-03T18:41:47.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:41:47.706+0000 7f6416052700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6410103fb0 0x7f6410103910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:41:47.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:41:47.706+0000 7f6416052700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f64101068e0 0x7f6410101f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:41:47.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:41:47.706+0000 7f6416052700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6410109210 0x7f64101024a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:41:47.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:41:47.706+0000 7f6416052700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f641010f030 con 0x7f6410103fb0 2024-05-03T18:41:47.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:41:47.706+0000 7f6416052700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f641010eeb0 con 0x7f6410109210 2024-05-03T18:41:47.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:41:47.706+0000 7f6416052700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f641010f1b0 con 0x7f64101068e0 2024-05-03T18:41:47.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:41:47.706+0000 7f640ffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6410109210 0x7f64101024a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:41:47.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:41:47.706+0000 7f640ffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6410109210 0x7f64101024a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39758/0 (socket says 172.21.15.45:39758) 2024-05-03T18:41:47.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:41:47.706+0000 7f640ffff700 1 -- 172.21.15.45:0/1448652758 learned_addr learned my addr 172.21.15.45:0/1448652758 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-03T18:41:47.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:41:47.706+0000 7f640effd700 1 --2- 172.21.15.45:0/1448652758 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f64101068e0 0x7f6410101f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:41:47.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:41:47.706+0000 7f640f7fe700 1 --2- 172.21.15.45:0/1448652758 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6410103fb0 0x7f6410103910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:41:47.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:41:47.706+0000 7f640ffff700 1 -- 172.21.15.45:0/1448652758 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f64101068e0 msgr2=0x7f6410101f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:41:47.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:41:47.706+0000 7f640ffff700 1 --2- 172.21.15.45:0/1448652758 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f64101068e0 0x7f6410101f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:47.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:41:47.706+0000 7f640ffff700 1 -- 172.21.15.45:0/1448652758 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6410103fb0 msgr2=0x7f6410103910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:41:47.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:41:47.706+0000 7f640ffff700 1 --2- 172.21.15.45:0/1448652758 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6410103fb0 0x7f6410103910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:41:47.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:41:47.706+0000 7f640ffff700 1 -- 172.21.15.45:0/1448652758 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6410102d50 con 0x7f6410109210 2024-05-03T18:41:47.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:41:47.706+0000 7f640effd700 1 --2- 172.21.15.45:0/1448652758 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f64101068e0 0x7f6410101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T18:41:47.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:41:47.706+0000 7f640f7fe700 1 --2- 172.21.15.45:0/1448652758 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6410103fb0 0x7f6410103910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:41:47.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:41:47.706+0000 7f640ffff700 1 --2- 172.21.15.45:0/1448652758 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6410109210 0x7f64101024a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f64080094f0 tx=0x7f6408007f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:41:47.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:41:47.706+0000 7f640cff9700 1 -- 172.21.15.45:0/1448652758 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6408004db0 con 0x7f6410109210 2024-05-03T18:41:47.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:41:47.707+0000 7f6416052700 1 -- 172.21.15.45:0/1448652758 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64101a37e0 con 0x7f6410109210 2024-05-03T18:41:47.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:41:47.707+0000 7f640cff9700 1 -- 172.21.15.45:0/1448652758 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f640800db60 con 0x7f6410109210 2024-05-03T18:41:47.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:41:47.707+0000 7f640cff9700 1 -- 172.21.15.45:0/1448652758 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6408015400 con 0x7f6410109210 2024-05-03T18:41:47.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:41:47.707+0000 7f6416052700 1 -- 172.21.15.45:0/1448652758 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64101a3d20 con 0x7f6410109210 2024-05-03T18:41:47.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:41:47.708+0000 7f6416052700 1 -- 172.21.15.45:0/1448652758 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f641006a190 con 0x7f6410109210 2024-05-03T18:41:47.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:41:47.709+0000 7f640cff9700 1 -- 172.21.15.45:0/1448652758 <== mon.2 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f640800dd10 con 0x7f6410109210 2024-05-03T18:41:47.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:41:47.710+0000 7f640cff9700 1 --2- 172.21.15.45:0/1448652758 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f63f80b2880 0x7f63f80b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:41:47.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:41:47.710+0000 7f640cff9700 1 -- 172.21.15.45:0/1448652758 <== mon.2 v2:172.21.15.45:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 4965+0+0 (secure 0 0 0) 0x7f64080e1850 con 0x7f6410109210 2024-05-03T18:41:47.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:41:47.710+0000 7f640f7fe700 1 --2- 172.21.15.45:0/1448652758 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f63f80b2880 0x7f63f80b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:41:47.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:41:47.711+0000 7f640f7fe700 1 --2- 172.21.15.45:0/1448652758 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f63f80b2880 0x7f63f80b4d40 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f6400006010 tx=0x7f6400003860 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:41:47.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:41:47.713+0000 7f640cff9700 1 -- 172.21.15.45:0/1448652758 <== mon.2 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f6408010070 con 0x7f6410109210 2024-05-03T18:41:47.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:41:47 smithi113 ceph-mon[33540]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-05-03T18:41:47.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:41:47 smithi113 ceph-mon[33540]: Detected new or changed devices on smithi032 2024-05-03T18:41:47.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:41:47 smithi113 ceph-mon[33540]: Adjusting osd_memory_target on smithi032 to 5697M 2024-05-03T18:41:47.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:41:47 smithi113 ceph-mon[33540]: osdmap e43: 7 total, 7 up, 7 in 2024-05-03T18:41:47.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:41:47 smithi032 ceph-mon[29763]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-05-03T18:41:47.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:41:47 smithi032 ceph-mon[29763]: Detected new or changed devices on smithi032 2024-05-03T18:41:47.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:41:47 smithi032 ceph-mon[29763]: Adjusting osd_memory_target on smithi032 to 5697M 2024-05-03T18:41:47.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:41:47 smithi032 ceph-mon[29763]: osdmap e43: 7 total, 7 up, 7 in 2024-05-03T18:41:47.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:41:47.935+0000 7f6416052700 1 -- 172.21.15.45:0/1448652758 --> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f64100638e0 con 0x7f63f80b2880 2024-05-03T18:41:48.680 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:41:48 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T18:41:48.680 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:41:48 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T18:41:48.680 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:41:48 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:41:48.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:41:48 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T18:41:48.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:41:48 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T18:41:48.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:41:48 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:41:48.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:41:48 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T18:41:48.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:41:48 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T18:41:48.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:41:48 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:41:49.495 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:41:49 smithi045 ceph-mon[33299]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-05-03T18:41:49.495 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:41:49 smithi045 ceph-mon[33299]: from='client.24248 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:41:49.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:41:49 smithi113 ceph-mon[33540]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-05-03T18:41:49.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:41:49 smithi113 ceph-mon[33540]: from='client.24248 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:41:49.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:41:49 smithi032 ceph-mon[29763]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-05-03T18:41:49.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:41:49 smithi032 ceph-mon[29763]: from='client.24248 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:41:51.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:41:51 smithi045 ceph-mon[33299]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-05-03T18:41:51.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:41:51 smithi045 ceph-mon[33299]: from='client.? 172.21.15.45:0/2296947997' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "988cce65-23ed-4814-ac09-177eb2f89fe1"}]: dispatch 2024-05-03T18:41:51.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:41:51 smithi045 ceph-mon[33299]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "988cce65-23ed-4814-ac09-177eb2f89fe1"}]: dispatch 2024-05-03T18:41:51.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:41:51 smithi045 ceph-mon[33299]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "988cce65-23ed-4814-ac09-177eb2f89fe1"}]': finished 2024-05-03T18:41:51.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:41:51 smithi045 ceph-mon[33299]: osdmap e44: 8 total, 7 up, 8 in 2024-05-03T18:41:51.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:41:51 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-03T18:41:51.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:41:51 smithi045 ceph-mon[33299]: from='client.? 172.21.15.45:0/731668237' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T18:41:51.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:41:51 smithi113 ceph-mon[33540]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-05-03T18:41:51.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:41:51 smithi113 ceph-mon[33540]: from='client.? 172.21.15.45:0/2296947997' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "988cce65-23ed-4814-ac09-177eb2f89fe1"}]: dispatch 2024-05-03T18:41:51.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:41:51 smithi113 ceph-mon[33540]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "988cce65-23ed-4814-ac09-177eb2f89fe1"}]: dispatch 2024-05-03T18:41:51.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:41:51 smithi113 ceph-mon[33540]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "988cce65-23ed-4814-ac09-177eb2f89fe1"}]': finished 2024-05-03T18:41:51.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:41:51 smithi113 ceph-mon[33540]: osdmap e44: 8 total, 7 up, 8 in 2024-05-03T18:41:51.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:41:51 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-03T18:41:51.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:41:51 smithi113 ceph-mon[33540]: from='client.? 172.21.15.45:0/731668237' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T18:41:51.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:41:51 smithi032 ceph-mon[29763]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-05-03T18:41:51.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:41:51 smithi032 ceph-mon[29763]: from='client.? 172.21.15.45:0/2296947997' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "988cce65-23ed-4814-ac09-177eb2f89fe1"}]: dispatch 2024-05-03T18:41:51.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:41:51 smithi032 ceph-mon[29763]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "988cce65-23ed-4814-ac09-177eb2f89fe1"}]: dispatch 2024-05-03T18:41:51.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:41:51 smithi032 ceph-mon[29763]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "988cce65-23ed-4814-ac09-177eb2f89fe1"}]': finished 2024-05-03T18:41:51.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:41:51 smithi032 ceph-mon[29763]: osdmap e44: 8 total, 7 up, 8 in 2024-05-03T18:41:51.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:41:51 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-03T18:41:51.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:41:51 smithi032 ceph-mon[29763]: from='client.? 172.21.15.45:0/731668237' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T18:41:53.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:41:53 smithi045 ceph-mon[33299]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-05-03T18:41:53.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:41:53 smithi113 ceph-mon[33540]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-05-03T18:41:53.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:41:53 smithi032 ceph-mon[29763]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-05-03T18:41:55.608 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:41:55 smithi045 ceph-mon[33299]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-05-03T18:41:55.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:41:55 smithi113 ceph-mon[33540]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-05-03T18:41:55.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:41:55 smithi032 ceph-mon[29763]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-05-03T18:41:57.624 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:41:57 smithi045 ceph-mon[33299]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-05-03T18:41:57.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:41:57 smithi113 ceph-mon[33540]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-05-03T18:41:57.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:41:57 smithi032 ceph-mon[29763]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-05-03T18:41:58.709 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:41:58 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-03T18:41:58.709 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:41:58 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:41:58.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:41:58 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-03T18:41:58.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:41:58 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:41:58.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:41:58 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-03T18:41:58.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:41:58 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:41:59.681 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:41:59 smithi045 ceph-mon[33299]: Deploying daemon osd.7 on smithi045 2024-05-03T18:41:59.681 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:41:59 smithi045 ceph-mon[33299]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-05-03T18:41:59.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:41:59 smithi113 ceph-mon[33540]: Deploying daemon osd.7 on smithi045 2024-05-03T18:41:59.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:41:59 smithi113 ceph-mon[33540]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-05-03T18:41:59.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:41:59 smithi032 ceph-mon[29763]: Deploying daemon osd.7 on smithi045 2024-05-03T18:41:59.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:41:59 smithi032 ceph-mon[29763]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-05-03T18:42:01.630 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:42:01 smithi045 ceph-mon[33299]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-05-03T18:42:01.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:42:01 smithi113 ceph-mon[33540]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-05-03T18:42:01.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:42:01 smithi032 ceph-mon[29763]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-05-03T18:42:03.747 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:42:03 smithi045 ceph-mon[33299]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-05-03T18:42:03.800 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:42:03 smithi113 ceph-mon[33540]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-05-03T18:42:03.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:42:03 smithi032 ceph-mon[29763]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-05-03T18:42:04.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:42:04 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:42:04.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:42:04 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:42:04.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:42:04 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:42:04.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:42:04 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:42:04.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:42:04 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:42:04.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:42:04 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:42:04.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:42:04 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:42:04.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:42:04 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:42:04.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:42:04 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:42:05.719 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:42:05 smithi032 ceph-mon[29763]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-05-03T18:42:05.747 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:42:05 smithi045 ceph-mon[33299]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-05-03T18:42:05.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:42:05 smithi113 ceph-mon[33540]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-05-03T18:42:07.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:42:07.203+0000 7f640cff9700 1 -- 172.21.15.45:0/1448652758 <== mgr.14152 v2:172.21.15.32:6800/1796547904 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f64100638e0 con 0x7f63f80b2880 2024-05-03T18:42:07.205 INFO:teuthology.orchestra.run.smithi045.stdout:Created osd(s) 7 on host 'smithi045' 2024-05-03T18:42:07.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:42:07.205+0000 7f6416052700 1 -- 172.21.15.45:0/1448652758 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f63f80b2880 msgr2=0x7f63f80b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:42:07.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:42:07.205+0000 7f6416052700 1 --2- 172.21.15.45:0/1448652758 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f63f80b2880 0x7f63f80b4d40 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f6400006010 tx=0x7f6400003860 comp rx=0 tx=0).stop 2024-05-03T18:42:07.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:42:07.205+0000 7f6416052700 1 -- 172.21.15.45:0/1448652758 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6410109210 msgr2=0x7f64101024a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:42:07.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:42:07.205+0000 7f6416052700 1 --2- 172.21.15.45:0/1448652758 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6410109210 0x7f64101024a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f64080094f0 tx=0x7f6408007f90 comp rx=0 tx=0).stop 2024-05-03T18:42:07.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:42:07.205+0000 7f6416052700 1 -- 172.21.15.45:0/1448652758 shutdown_connections 2024-05-03T18:42:07.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:42:07.205+0000 7f6416052700 1 --2- 172.21.15.45:0/1448652758 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f63f80b2880 0x7f63f80b4d40 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:07.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:42:07.205+0000 7f6416052700 1 --2- 172.21.15.45:0/1448652758 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6410109210 0x7f64101024a0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:07.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:42:07.205+0000 7f6416052700 1 --2- 172.21.15.45:0/1448652758 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6410103fb0 0x7f6410103910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:07.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:42:07.205+0000 7f6416052700 1 --2- 172.21.15.45:0/1448652758 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f64101068e0 0x7f6410101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:07.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:42:07.205+0000 7f6416052700 1 -- 172.21.15.45:0/1448652758 >> 172.21.15.45:0/1448652758 conn(0x7f64100fd9c0 msgr2=0x7f6410104a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:42:07.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:42:07.205+0000 7f6416052700 1 -- 172.21.15.45:0/1448652758 shutdown_connections 2024-05-03T18:42:07.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:42:07.205+0000 7f6416052700 1 -- 172.21.15.45:0/1448652758 wait complete. 2024-05-03T18:42:07.497 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:42:07 smithi045 ceph-mon[33299]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-05-03T18:42:07.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:42:07 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:42:07.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:42:07 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:42:07.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:42:07 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:42:07.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:42:07 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:42:07.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:42:07 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:42:07.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:42:07 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:42:07.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:42:07 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:42:07.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:42:07 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:42:07.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:42:07 smithi045 ceph-mon[33299]: from='osd.7 [v2:172.21.15.45:6816/1032572438,v1:172.21.15.45:6817/1032572438]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-03T18:42:07.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:42:07 smithi045 ceph-mon[33299]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-03T18:42:07.776 DEBUG:teuthology.orchestra.run.smithi045:osd.7> sudo journalctl -f -n 0 -u ceph-f500455e-097b-11ef-bc97-c7b262605968@osd.7.service 2024-05-03T18:42:07.778 INFO:tasks.cephadm:Deploying osd.8 on smithi113 with /dev/vg_nvme/lv_2... 2024-05-03T18:42:07.778 DEBUG:teuthology.orchestra.run.smithi113:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f500455e-097b-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-03T18:42:07.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:42:07 smithi113 ceph-mon[33540]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-05-03T18:42:07.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:42:07 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:42:07.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:42:07 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:42:07.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:42:07 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:42:07.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:42:07 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:42:07.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:42:07 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:42:07.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:42:07 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:42:07.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:42:07 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:42:07.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:42:07 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:42:07.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:42:07 smithi113 ceph-mon[33540]: from='osd.7 [v2:172.21.15.45:6816/1032572438,v1:172.21.15.45:6817/1032572438]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-03T18:42:07.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:42:07 smithi113 ceph-mon[33540]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-03T18:42:07.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:42:07 smithi032 ceph-mon[29763]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-05-03T18:42:07.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:42:07 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:42:07.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:42:07 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:42:07.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:42:07 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:42:07.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:42:07 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:42:07.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:42:07 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:42:07.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:42:07 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:42:07.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:42:07 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:42:07.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:42:07 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:42:07.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:42:07 smithi032 ceph-mon[29763]: from='osd.7 [v2:172.21.15.45:6816/1032572438,v1:172.21.15.45:6817/1032572438]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-03T18:42:07.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:42:07 smithi032 ceph-mon[29763]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-03T18:42:08.022 INFO:teuthology.orchestra.run.smithi113.stderr:Inferring config /var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/mon.c/config 2024-05-03T18:42:09.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:42:09 smithi113 ceph-mon[33540]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-05-03T18:42:09.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:42:09 smithi113 ceph-mon[33540]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-03T18:42:09.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:42:09 smithi113 ceph-mon[33540]: from='osd.7 [v2:172.21.15.45:6816/1032572438,v1:172.21.15.45:6817/1032572438]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-05-03T18:42:09.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:42:09 smithi113 ceph-mon[33540]: osdmap e45: 8 total, 7 up, 8 in 2024-05-03T18:42:09.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:42:09 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-03T18:42:09.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:42:09 smithi113 ceph-mon[33540]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-05-03T18:42:09.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:42:09 smithi045 ceph-mon[33299]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-05-03T18:42:09.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:42:09 smithi045 ceph-mon[33299]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-03T18:42:09.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:42:09 smithi045 ceph-mon[33299]: from='osd.7 [v2:172.21.15.45:6816/1032572438,v1:172.21.15.45:6817/1032572438]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-05-03T18:42:09.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:42:09 smithi045 ceph-mon[33299]: osdmap e45: 8 total, 7 up, 8 in 2024-05-03T18:42:09.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:42:09 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-03T18:42:09.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:42:09 smithi045 ceph-mon[33299]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-05-03T18:42:09.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:42:09 smithi032 ceph-mon[29763]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-05-03T18:42:09.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:42:09 smithi032 ceph-mon[29763]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-03T18:42:09.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:42:09 smithi032 ceph-mon[29763]: from='osd.7 [v2:172.21.15.45:6816/1032572438,v1:172.21.15.45:6817/1032572438]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-05-03T18:42:09.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:42:09 smithi032 ceph-mon[29763]: osdmap e45: 8 total, 7 up, 8 in 2024-05-03T18:42:09.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:42:09 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-03T18:42:09.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:42:09 smithi032 ceph-mon[29763]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-05-03T18:42:09.857 INFO:teuthology.orchestra.run.smithi113.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-03T18:42:09.857 INFO:teuthology.orchestra.run.smithi113.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-03T18:42:09.858 INFO:teuthology.orchestra.run.smithi113.stderr: stderr: 10+0 records in 2024-05-03T18:42:09.858 INFO:teuthology.orchestra.run.smithi113.stderr:10+0 records out 2024-05-03T18:42:09.858 INFO:teuthology.orchestra.run.smithi113.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00951504 s, 1.1 GB/s 2024-05-03T18:42:09.858 INFO:teuthology.orchestra.run.smithi113.stderr:--> Zapping successful for: 2024-05-03T18:42:10.175 INFO:journalctl@ceph.osd.7.smithi045.stdout:May 03 18:42:09 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-7[51710]: 2024-05-03T18:42:09.950+0000 7fe2c6ba6700 -1 osd.7 0 waiting for initial osdmap 2024-05-03T18:42:10.175 INFO:journalctl@ceph.osd.7.smithi045.stdout:May 03 18:42:09 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-7[51710]: 2024-05-03T18:42:09.963+0000 7fe2be991700 -1 osd.7 46 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-03T18:42:10.467 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:42:10 smithi113 ceph-mon[33540]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2024-05-03T18:42:10.467 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:42:10 smithi113 ceph-mon[33540]: osdmap e46: 8 total, 7 up, 8 in 2024-05-03T18:42:10.467 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:42:10 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-03T18:42:10.467 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:42:10 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-03T18:42:10.467 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:42:10 smithi113 ceph-mon[33540]: from='osd.7 ' entity='osd.7' 2024-05-03T18:42:10.476 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:42:10 smithi045 ceph-mon[33299]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2024-05-03T18:42:10.476 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:42:10 smithi045 ceph-mon[33299]: osdmap e46: 8 total, 7 up, 8 in 2024-05-03T18:42:10.476 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:42:10 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-03T18:42:10.476 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:42:10 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-03T18:42:10.476 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:42:10 smithi045 ceph-mon[33299]: from='osd.7 ' entity='osd.7' 2024-05-03T18:42:10.495 DEBUG:teuthology.orchestra.run.smithi113:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f500455e-097b-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi113:vg_nvme/lv_2 2024-05-03T18:42:10.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:42:10 smithi032 ceph-mon[29763]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2024-05-03T18:42:10.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:42:10 smithi032 ceph-mon[29763]: osdmap e46: 8 total, 7 up, 8 in 2024-05-03T18:42:10.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:42:10 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-03T18:42:10.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:42:10 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-03T18:42:10.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:42:10 smithi032 ceph-mon[29763]: from='osd.7 ' entity='osd.7' 2024-05-03T18:42:10.780 INFO:teuthology.orchestra.run.smithi113.stderr:Inferring config /var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/mon.c/config 2024-05-03T18:42:11.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:42:11 smithi113 ceph-mon[33540]: purged_snaps scrub starts 2024-05-03T18:42:11.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:42:11 smithi113 ceph-mon[33540]: purged_snaps scrub ok 2024-05-03T18:42:11.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:42:11 smithi113 ceph-mon[33540]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-05-03T18:42:11.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:42:11 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-03T18:42:11.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:42:11 smithi113 ceph-mon[33540]: osd.7 [v2:172.21.15.45:6816/1032572438,v1:172.21.15.45:6817/1032572438] boot 2024-05-03T18:42:11.302 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:42:11 smithi113 ceph-mon[33540]: osdmap e47: 8 total, 8 up, 8 in 2024-05-03T18:42:11.302 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:42:11 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-03T18:42:11.302 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:42:11 smithi113 ceph-mon[33540]: Detected new or changed devices on smithi045 2024-05-03T18:42:11.302 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:42:11 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:42:11.302 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:42:11 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:42:11.302 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:42:11 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:42:11.302 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:42:11 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:42:11.302 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:42:11 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:42:11.302 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:42:11 smithi113 ceph-mon[33540]: Adjusting osd_memory_target on smithi045 to 5697M 2024-05-03T18:42:11.302 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:42:11 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:42:11.302 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:42:11 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:42:11.302 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:42:11 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:42:11.302 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:42:11 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:42:11.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:42:11 smithi045 ceph-mon[33299]: purged_snaps scrub starts 2024-05-03T18:42:11.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:42:11 smithi045 ceph-mon[33299]: purged_snaps scrub ok 2024-05-03T18:42:11.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:42:11 smithi045 ceph-mon[33299]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-05-03T18:42:11.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:42:11 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-03T18:42:11.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:42:11 smithi045 ceph-mon[33299]: osd.7 [v2:172.21.15.45:6816/1032572438,v1:172.21.15.45:6817/1032572438] boot 2024-05-03T18:42:11.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:42:11 smithi045 ceph-mon[33299]: osdmap e47: 8 total, 8 up, 8 in 2024-05-03T18:42:11.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:42:11 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-03T18:42:11.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:42:11 smithi045 ceph-mon[33299]: Detected new or changed devices on smithi045 2024-05-03T18:42:11.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:42:11 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:42:11.499 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:42:11 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:42:11.499 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:42:11 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:42:11.499 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:42:11 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:42:11.499 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:42:11 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:42:11.499 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:42:11 smithi045 ceph-mon[33299]: Adjusting osd_memory_target on smithi045 to 5697M 2024-05-03T18:42:11.499 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:42:11 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:42:11.499 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:42:11 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:42:11.499 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:42:11 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:42:11.499 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:42:11 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:42:11.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:42:11 smithi032 ceph-mon[29763]: purged_snaps scrub starts 2024-05-03T18:42:11.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:42:11 smithi032 ceph-mon[29763]: purged_snaps scrub ok 2024-05-03T18:42:11.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:42:11 smithi032 ceph-mon[29763]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-05-03T18:42:11.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:42:11 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-03T18:42:11.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:42:11 smithi032 ceph-mon[29763]: osd.7 [v2:172.21.15.45:6816/1032572438,v1:172.21.15.45:6817/1032572438] boot 2024-05-03T18:42:11.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:42:11 smithi032 ceph-mon[29763]: osdmap e47: 8 total, 8 up, 8 in 2024-05-03T18:42:11.569 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:42:11 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-03T18:42:11.569 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:42:11 smithi032 ceph-mon[29763]: Detected new or changed devices on smithi045 2024-05-03T18:42:11.569 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:42:11 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:42:11.569 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:42:11 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:42:11.569 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:42:11 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:42:11.569 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:42:11 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:42:11.569 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:42:11 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:42:11.569 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:42:11 smithi032 ceph-mon[29763]: Adjusting osd_memory_target on smithi045 to 5697M 2024-05-03T18:42:11.569 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:42:11 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:42:11.569 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:42:11 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:42:11.569 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:42:11 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:42:11.569 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:42:11 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:42:12.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:42:12.296+0000 7ff94eb18700 1 -- 172.21.15.113:0/2902273873 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff948102420 msgr2=0x7ff948102840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:42:12.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:42:12.296+0000 7ff94eb18700 1 --2- 172.21.15.113:0/2902273873 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff948102420 0x7ff948102840 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7ff93c009990 tx=0x7ff93c009ca0 comp rx=0 tx=0).stop 2024-05-03T18:42:12.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:42:12.297+0000 7ff94eb18700 1 -- 172.21.15.113:0/2902273873 shutdown_connections 2024-05-03T18:42:12.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:42:12.297+0000 7ff94eb18700 1 --2- 172.21.15.113:0/2902273873 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff948103fc0 0x7ff94810a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:12.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:42:12.297+0000 7ff94eb18700 1 --2- 172.21.15.113:0/2902273873 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff948102420 0x7ff948102840 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:12.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:42:12.297+0000 7ff94eb18700 1 --2- 172.21.15.113:0/2902273873 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff948103620 0x7ff948103a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:12.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:42:12.297+0000 7ff94eb18700 1 -- 172.21.15.113:0/2902273873 >> 172.21.15.113:0/2902273873 conn(0x7ff9480fd9c0 msgr2=0x7ff9480ffe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:42:12.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:42:12.297+0000 7ff94eb18700 1 -- 172.21.15.113:0/2902273873 shutdown_connections 2024-05-03T18:42:12.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:42:12.297+0000 7ff94eb18700 1 -- 172.21.15.113:0/2902273873 wait complete. 2024-05-03T18:42:12.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:42:12.297+0000 7ff94eb18700 1 Processor -- start 2024-05-03T18:42:12.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:42:12.297+0000 7ff94eb18700 1 -- start start 2024-05-03T18:42:12.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:42:12.297+0000 7ff94eb18700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff948102420 0x7ff948199780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:42:12.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:42:12.297+0000 7ff94eb18700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff948103620 0x7ff948199cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:42:12.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:42:12.297+0000 7ff94eb18700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff948103fc0 0x7ff9481a0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:42:12.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:42:12.297+0000 7ff94eb18700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff94810df70 con 0x7ff948103fc0 2024-05-03T18:42:12.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:42:12.297+0000 7ff94eb18700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff94810ddf0 con 0x7ff948102420 2024-05-03T18:42:12.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:42:12.297+0000 7ff94eb18700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7ff94810e0f0 con 0x7ff948103620 2024-05-03T18:42:12.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:42:12.298+0000 7ff947fff700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff948103620 0x7ff948199cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:42:12.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:42:12.298+0000 7ff947fff700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff948103620 0x7ff948199cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:43318/0 (socket says 172.21.15.113:43318) 2024-05-03T18:42:12.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:42:12.298+0000 7ff947fff700 1 -- 172.21.15.113:0/3997646841 learned_addr learned my addr 172.21.15.113:0/3997646841 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-05-03T18:42:12.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:42:12.298+0000 7ff94d0b5700 1 --2- 172.21.15.113:0/3997646841 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff948103fc0 0x7ff9481a0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:42:12.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:42:12.298+0000 7ff94c8b4700 1 --2- 172.21.15.113:0/3997646841 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff948102420 0x7ff948199780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:42:12.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:42:12.298+0000 7ff947fff700 1 -- 172.21.15.113:0/3997646841 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff948102420 msgr2=0x7ff948199780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:42:12.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:42:12.298+0000 7ff947fff700 1 --2- 172.21.15.113:0/3997646841 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff948102420 0x7ff948199780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:12.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:42:12.298+0000 7ff947fff700 1 -- 172.21.15.113:0/3997646841 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff948103fc0 msgr2=0x7ff9481a0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:42:12.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:42:12.298+0000 7ff947fff700 1 --2- 172.21.15.113:0/3997646841 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff948103fc0 0x7ff9481a0cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:12.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:42:12.298+0000 7ff947fff700 1 -- 172.21.15.113:0/3997646841 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9481a1370 con 0x7ff948103620 2024-05-03T18:42:12.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:42:12.298+0000 7ff94c8b4700 1 --2- 172.21.15.113:0/3997646841 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff948102420 0x7ff948199780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:42:12.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:42:12.298+0000 7ff94d0b5700 1 --2- 172.21.15.113:0/3997646841 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff948103fc0 0x7ff9481a0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T18:42:12.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:42:12.298+0000 7ff947fff700 1 --2- 172.21.15.113:0/3997646841 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff948103620 0x7ff948199cc0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7ff93400d980 tx=0x7ff93400dd40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:42:12.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:42:12.298+0000 7ff945ffb700 1 -- 172.21.15.113:0/3997646841 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff934016070 con 0x7ff948103620 2024-05-03T18:42:12.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:42:12.298+0000 7ff945ffb700 1 -- 172.21.15.113:0/3997646841 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff934010460 con 0x7ff948103620 2024-05-03T18:42:12.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:42:12.298+0000 7ff94eb18700 1 -- 172.21.15.113:0/3997646841 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9481a1660 con 0x7ff948103620 2024-05-03T18:42:12.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:42:12.298+0000 7ff945ffb700 1 -- 172.21.15.113:0/3997646841 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff93400f620 con 0x7ff948103620 2024-05-03T18:42:12.302 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:42:12.298+0000 7ff94eb18700 1 -- 172.21.15.113:0/3997646841 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9481a1ba0 con 0x7ff948103620 2024-05-03T18:42:12.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:42:12.299+0000 7ff94eb18700 1 -- 172.21.15.113:0/3997646841 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff94806a190 con 0x7ff948103620 2024-05-03T18:42:12.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:42:12.300+0000 7ff945ffb700 1 -- 172.21.15.113:0/3997646841 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7ff934019070 con 0x7ff948103620 2024-05-03T18:42:12.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:42:12.301+0000 7ff945ffb700 1 --2- 172.21.15.113:0/3997646841 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7ff9380b2880 0x7ff9380b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:42:12.303 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:42:12.301+0000 7ff945ffb700 1 -- 172.21.15.113:0/3997646841 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5397+0+0 (secure 0 0 0) 0x7ff9340e12a0 con 0x7ff948103620 2024-05-03T18:42:12.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:42:12.301+0000 7ff94c8b4700 1 --2- 172.21.15.113:0/3997646841 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7ff9380b2880 0x7ff9380b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:42:12.304 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:42:12.302+0000 7ff94c8b4700 1 --2- 172.21.15.113:0/3997646841 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7ff9380b2880 0x7ff9380b4d40 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff93c006010 tx=0x7ff93c003860 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:42:12.305 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:42:12.305+0000 7ff945ffb700 1 -- 172.21.15.113:0/3997646841 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7ff9340a9b20 con 0x7ff948103620 2024-05-03T18:42:12.497 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:42:12 smithi045 ceph-mon[33299]: osdmap e48: 8 total, 8 up, 8 in 2024-05-03T18:42:12.519 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:42:12.519+0000 7ff94eb18700 1 -- 172.21.15.113:0/3997646841 --> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi113:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7ff9480ffba0 con 0x7ff9380b2880 2024-05-03T18:42:12.519 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:42:12 smithi113 ceph-mon[33540]: osdmap e48: 8 total, 8 up, 8 in 2024-05-03T18:42:12.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:42:12 smithi032 ceph-mon[29763]: osdmap e48: 8 total, 8 up, 8 in 2024-05-03T18:42:13.449 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:42:13 smithi113 ceph-mon[33540]: pgmap v171: 1 pgs: 1 remapped; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-05-03T18:42:13.450 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:42:13 smithi113 ceph-mon[33540]: osdmap e49: 8 total, 8 up, 8 in 2024-05-03T18:42:13.450 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:42:13 smithi113 ceph-mon[33540]: from='client.24313 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi113:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:42:13.450 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:42:13 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T18:42:13.450 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:42:13 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T18:42:13.450 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:42:13 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:42:13.497 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:42:13 smithi045 ceph-mon[33299]: pgmap v171: 1 pgs: 1 remapped; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-05-03T18:42:13.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:42:13 smithi045 ceph-mon[33299]: osdmap e49: 8 total, 8 up, 8 in 2024-05-03T18:42:13.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:42:13 smithi045 ceph-mon[33299]: from='client.24313 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi113:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:42:13.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:42:13 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T18:42:13.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:42:13 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T18:42:13.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:42:13 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:42:13.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:42:13 smithi032 ceph-mon[29763]: pgmap v171: 1 pgs: 1 remapped; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-05-03T18:42:13.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:42:13 smithi032 ceph-mon[29763]: osdmap e49: 8 total, 8 up, 8 in 2024-05-03T18:42:13.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:42:13 smithi032 ceph-mon[29763]: from='client.24313 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi113:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:42:13.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:42:13 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T18:42:13.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:42:13 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T18:42:13.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:42:13 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:42:15.497 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:42:15 smithi045 ceph-mon[33299]: pgmap v173: 1 pgs: 1 remapped; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-05-03T18:42:15.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:42:15 smithi113 ceph-mon[33540]: pgmap v173: 1 pgs: 1 remapped; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-05-03T18:42:15.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:42:15 smithi032 ceph-mon[29763]: pgmap v173: 1 pgs: 1 remapped; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-05-03T18:42:16.471 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:42:16 smithi032 ceph-mon[29763]: from='client.? 172.21.15.113:0/806984250' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "816e8ce8-5afa-4b52-89be-f7793ed12a0b"}]: dispatch 2024-05-03T18:42:16.472 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:42:16 smithi032 ceph-mon[29763]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "816e8ce8-5afa-4b52-89be-f7793ed12a0b"}]: dispatch 2024-05-03T18:42:16.472 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:42:16 smithi032 ceph-mon[29763]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "816e8ce8-5afa-4b52-89be-f7793ed12a0b"}]': finished 2024-05-03T18:42:16.472 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:42:16 smithi032 ceph-mon[29763]: osdmap e50: 9 total, 8 up, 9 in 2024-05-03T18:42:16.472 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:42:16 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-03T18:42:16.472 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:42:16 smithi032 ceph-mon[29763]: from='client.? 172.21.15.113:0/2398755287' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T18:42:16.497 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:42:16 smithi045 ceph-mon[33299]: from='client.? 172.21.15.113:0/806984250' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "816e8ce8-5afa-4b52-89be-f7793ed12a0b"}]: dispatch 2024-05-03T18:42:16.497 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:42:16 smithi045 ceph-mon[33299]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "816e8ce8-5afa-4b52-89be-f7793ed12a0b"}]: dispatch 2024-05-03T18:42:16.497 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:42:16 smithi045 ceph-mon[33299]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "816e8ce8-5afa-4b52-89be-f7793ed12a0b"}]': finished 2024-05-03T18:42:16.497 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:42:16 smithi045 ceph-mon[33299]: osdmap e50: 9 total, 8 up, 9 in 2024-05-03T18:42:16.497 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:42:16 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-03T18:42:16.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:42:16 smithi045 ceph-mon[33299]: from='client.? 172.21.15.113:0/2398755287' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T18:42:16.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:42:16 smithi113 ceph-mon[33540]: from='client.? 172.21.15.113:0/806984250' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "816e8ce8-5afa-4b52-89be-f7793ed12a0b"}]: dispatch 2024-05-03T18:42:16.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:42:16 smithi113 ceph-mon[33540]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "816e8ce8-5afa-4b52-89be-f7793ed12a0b"}]: dispatch 2024-05-03T18:42:16.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:42:16 smithi113 ceph-mon[33540]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "816e8ce8-5afa-4b52-89be-f7793ed12a0b"}]': finished 2024-05-03T18:42:16.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:42:16 smithi113 ceph-mon[33540]: osdmap e50: 9 total, 8 up, 9 in 2024-05-03T18:42:16.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:42:16 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-03T18:42:16.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:42:16 smithi113 ceph-mon[33540]: from='client.? 172.21.15.113:0/2398755287' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T18:42:17.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:42:17 smithi032 ceph-mon[29763]: pgmap v175: 1 pgs: 1 remapped; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-05-03T18:42:17.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:42:17 smithi113 ceph-mon[33540]: pgmap v175: 1 pgs: 1 remapped; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-05-03T18:42:17.747 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:42:17 smithi045 ceph-mon[33299]: pgmap v175: 1 pgs: 1 remapped; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-05-03T18:42:19.461 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:42:19 smithi113 ceph-mon[33540]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail; 171 KiB/s, 0 objects/s recovering 2024-05-03T18:42:19.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:42:19 smithi032 ceph-mon[29763]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail; 171 KiB/s, 0 objects/s recovering 2024-05-03T18:42:19.747 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:42:19 smithi045 ceph-mon[33299]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail; 171 KiB/s, 0 objects/s recovering 2024-05-03T18:42:21.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:42:21 smithi113 ceph-mon[33540]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-05-03T18:42:21.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:42:21 smithi032 ceph-mon[29763]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-05-03T18:42:21.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:42:21 smithi045 ceph-mon[33299]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-05-03T18:42:23.497 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:42:23 smithi113 ceph-mon[33540]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail; 118 KiB/s, 0 objects/s recovering 2024-05-03T18:42:23.498 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:42:23 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-03T18:42:23.498 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:42:23 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:42:23.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:42:23 smithi032 ceph-mon[29763]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail; 118 KiB/s, 0 objects/s recovering 2024-05-03T18:42:23.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:42:23 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-03T18:42:23.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:42:23 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:42:23.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:42:23 smithi045 ceph-mon[33299]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail; 118 KiB/s, 0 objects/s recovering 2024-05-03T18:42:23.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:42:23 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-03T18:42:23.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:42:23 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:42:24.458 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:42:24 smithi113 ceph-mon[33540]: Deploying daemon osd.8 on smithi113 2024-05-03T18:42:24.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:42:24 smithi032 ceph-mon[29763]: Deploying daemon osd.8 on smithi113 2024-05-03T18:42:24.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:42:24 smithi045 ceph-mon[33299]: Deploying daemon osd.8 on smithi113 2024-05-03T18:42:25.379 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:42:25 smithi113 ceph-mon[33540]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-05-03T18:42:25.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:42:25 smithi032 ceph-mon[29763]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-05-03T18:42:25.747 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:42:25 smithi045 ceph-mon[33299]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-05-03T18:42:27.407 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:42:27 smithi113 ceph-mon[33540]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail; 107 KiB/s, 0 objects/s recovering 2024-05-03T18:42:27.550 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:42:27 smithi045 ceph-mon[33299]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail; 107 KiB/s, 0 objects/s recovering 2024-05-03T18:42:27.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:42:27 smithi032 ceph-mon[29763]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail; 107 KiB/s, 0 objects/s recovering 2024-05-03T18:42:29.238 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:42:28 smithi113 ceph-mon[33540]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-03T18:42:29.238 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:42:28 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:42:29.238 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:42:28 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:42:29.238 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:42:28 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:42:29.247 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:42:28 smithi045 ceph-mon[33299]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-03T18:42:29.247 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:42:28 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:42:29.247 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:42:28 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:42:29.247 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:42:28 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:42:29.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:42:28 smithi032 ceph-mon[29763]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-03T18:42:29.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:42:28 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:42:29.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:42:28 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:42:29.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:42:28 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:42:31.247 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:42:30 smithi045 ceph-mon[33299]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-05-03T18:42:31.248 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:42:30 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:42:31.248 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:42:30 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:42:31.248 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:42:30 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:42:31.248 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:42:30 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:42:31.248 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:42:30 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:42:31.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:42:30 smithi113 ceph-mon[33540]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-05-03T18:42:31.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:42:30 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:42:31.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:42:30 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:42:31.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:42:30 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:42:31.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:42:30 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:42:31.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:42:30 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:42:31.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:42:30 smithi032 ceph-mon[29763]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-05-03T18:42:31.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:42:30 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:42:31.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:42:30 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:42:31.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:42:30 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:42:31.319 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:42:30 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:42:31.319 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:42:30 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:42:31.693 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:42:31.692+0000 7ff945ffb700 1 -- 172.21.15.113:0/3997646841 <== mgr.14152 v2:172.21.15.32:6800/1796547904 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7ff9480ffba0 con 0x7ff9380b2880 2024-05-03T18:42:31.693 INFO:teuthology.orchestra.run.smithi113.stdout:Created osd(s) 8 on host 'smithi113' 2024-05-03T18:42:31.696 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:42:31.695+0000 7ff94eb18700 1 -- 172.21.15.113:0/3997646841 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7ff9380b2880 msgr2=0x7ff9380b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:42:31.696 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:42:31.695+0000 7ff94eb18700 1 --2- 172.21.15.113:0/3997646841 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7ff9380b2880 0x7ff9380b4d40 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff93c006010 tx=0x7ff93c003860 comp rx=0 tx=0).stop 2024-05-03T18:42:31.696 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:42:31.695+0000 7ff94eb18700 1 -- 172.21.15.113:0/3997646841 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff948103620 msgr2=0x7ff948199cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:42:31.696 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:42:31.695+0000 7ff94eb18700 1 --2- 172.21.15.113:0/3997646841 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff948103620 0x7ff948199cc0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7ff93400d980 tx=0x7ff93400dd40 comp rx=0 tx=0).stop 2024-05-03T18:42:31.696 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:42:31.695+0000 7ff94eb18700 1 -- 172.21.15.113:0/3997646841 shutdown_connections 2024-05-03T18:42:31.696 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:42:31.695+0000 7ff94eb18700 1 --2- 172.21.15.113:0/3997646841 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7ff9380b2880 0x7ff9380b4d40 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:31.696 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:42:31.695+0000 7ff94eb18700 1 --2- 172.21.15.113:0/3997646841 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff948103fc0 0x7ff9481a0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:31.696 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:42:31.695+0000 7ff94eb18700 1 --2- 172.21.15.113:0/3997646841 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff948102420 0x7ff948199780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:31.696 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:42:31.695+0000 7ff94eb18700 1 --2- 172.21.15.113:0/3997646841 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff948103620 0x7ff948199cc0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:31.696 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:42:31.695+0000 7ff94eb18700 1 -- 172.21.15.113:0/3997646841 >> 172.21.15.113:0/3997646841 conn(0x7ff9480fd9c0 msgr2=0x7ff9480ff490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:42:31.696 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:42:31.695+0000 7ff94eb18700 1 -- 172.21.15.113:0/3997646841 shutdown_connections 2024-05-03T18:42:31.697 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:42:31.695+0000 7ff94eb18700 1 -- 172.21.15.113:0/3997646841 wait complete. 2024-05-03T18:42:32.359 DEBUG:teuthology.orchestra.run.smithi113:osd.8> sudo journalctl -f -n 0 -u ceph-f500455e-097b-11ef-bc97-c7b262605968@osd.8.service 2024-05-03T18:42:32.361 INFO:tasks.cephadm:Deploying osd.9 on smithi032 with /dev/vg_nvme/lv_1... 2024-05-03T18:42:32.362 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f500455e-097b-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-03T18:42:32.621 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:42:32.745 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:42:32 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:42:32.745 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:42:32 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:42:32.745 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:42:32 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:42:32.745 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:42:32 smithi032 ceph-mon[29763]: from='osd.8 [v2:172.21.15.113:6816/3569181558,v1:172.21.15.113:6817/3569181558]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-03T18:42:32.745 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:42:32 smithi032 ceph-mon[29763]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-03T18:42:32.745 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:42:32 smithi032 ceph-mon[29763]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-05-03T18:42:32.997 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:42:32 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:42:32.997 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:42:32 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:42:32.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:42:32 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:42:32.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:42:32 smithi045 ceph-mon[33299]: from='osd.8 [v2:172.21.15.113:6816/3569181558,v1:172.21.15.113:6817/3569181558]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-03T18:42:32.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:42:32 smithi045 ceph-mon[33299]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-03T18:42:32.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:42:32 smithi045 ceph-mon[33299]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-05-03T18:42:33.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:42:32 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:42:33.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:42:32 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:42:33.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:42:32 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:42:33.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:42:32 smithi113 ceph-mon[33540]: from='osd.8 [v2:172.21.15.113:6816/3569181558,v1:172.21.15.113:6817/3569181558]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-03T18:42:33.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:42:32 smithi113 ceph-mon[33540]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-03T18:42:33.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:42:32 smithi113 ceph-mon[33540]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-05-03T18:42:33.677 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:42:33 smithi113 ceph-mon[33540]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-03T18:42:33.677 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:42:33 smithi113 ceph-mon[33540]: osdmap e51: 9 total, 8 up, 9 in 2024-05-03T18:42:33.677 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:42:33 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-03T18:42:33.677 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:42:33 smithi113 ceph-mon[33540]: from='osd.8 [v2:172.21.15.113:6816/3569181558,v1:172.21.15.113:6817/3569181558]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi113", "root=default"]}]: dispatch 2024-05-03T18:42:33.677 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:42:33 smithi113 ceph-mon[33540]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi113", "root=default"]}]: dispatch 2024-05-03T18:42:33.677 INFO:journalctl@ceph.osd.8.smithi113.stdout:May 03 18:42:33 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-8[52667]: 2024-05-03T18:42:33.663+0000 7fb4ab83c700 -1 osd.8 0 waiting for initial osdmap 2024-05-03T18:42:33.677 INFO:journalctl@ceph.osd.8.smithi113.stdout:May 03 18:42:33 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-8[52667]: 2024-05-03T18:42:33.676+0000 7fb4a762f700 -1 osd.8 52 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-03T18:42:33.997 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:42:33 smithi045 ceph-mon[33299]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-03T18:42:33.997 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:42:33 smithi045 ceph-mon[33299]: osdmap e51: 9 total, 8 up, 9 in 2024-05-03T18:42:33.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:42:33 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-03T18:42:33.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:42:33 smithi045 ceph-mon[33299]: from='osd.8 [v2:172.21.15.113:6816/3569181558,v1:172.21.15.113:6817/3569181558]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi113", "root=default"]}]: dispatch 2024-05-03T18:42:33.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:42:33 smithi045 ceph-mon[33299]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi113", "root=default"]}]: dispatch 2024-05-03T18:42:34.012 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:42:33 smithi032 ceph-mon[29763]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-03T18:42:34.012 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:42:33 smithi032 ceph-mon[29763]: osdmap e51: 9 total, 8 up, 9 in 2024-05-03T18:42:34.012 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:42:33 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-03T18:42:34.012 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:42:33 smithi032 ceph-mon[29763]: from='osd.8 [v2:172.21.15.113:6816/3569181558,v1:172.21.15.113:6817/3569181558]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi113", "root=default"]}]: dispatch 2024-05-03T18:42:34.012 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:42:33 smithi032 ceph-mon[29763]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi113", "root=default"]}]: dispatch 2024-05-03T18:42:34.713 INFO:teuthology.orchestra.run.smithi032.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-03T18:42:34.713 INFO:teuthology.orchestra.run.smithi032.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-03T18:42:34.713 INFO:teuthology.orchestra.run.smithi032.stderr: stderr: 10+0 records in 2024-05-03T18:42:34.713 INFO:teuthology.orchestra.run.smithi032.stderr:10+0 records out 2024-05-03T18:42:34.714 INFO:teuthology.orchestra.run.smithi032.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00823503 s, 1.3 GB/s 2024-05-03T18:42:34.714 INFO:teuthology.orchestra.run.smithi032.stderr:--> Zapping successful for: 2024-05-03T18:42:34.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:42:34 smithi113 ceph-mon[33540]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi113", "root=default"]}]': finished 2024-05-03T18:42:34.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:42:34 smithi113 ceph-mon[33540]: osdmap e52: 9 total, 8 up, 9 in 2024-05-03T18:42:34.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:42:34 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-03T18:42:34.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:42:34 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-03T18:42:34.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:42:34 smithi113 ceph-mon[33540]: osd.8 [v2:172.21.15.113:6816/3569181558,v1:172.21.15.113:6817/3569181558] boot 2024-05-03T18:42:34.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:42:34 smithi113 ceph-mon[33540]: osdmap e53: 9 total, 9 up, 9 in 2024-05-03T18:42:34.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:42:34 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-03T18:42:34.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:42:34 smithi113 ceph-mon[33540]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-05-03T18:42:34.997 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:42:34 smithi045 ceph-mon[33299]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi113", "root=default"]}]': finished 2024-05-03T18:42:34.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:42:34 smithi045 ceph-mon[33299]: osdmap e52: 9 total, 8 up, 9 in 2024-05-03T18:42:34.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:42:34 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-03T18:42:34.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:42:34 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-03T18:42:34.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:42:34 smithi045 ceph-mon[33299]: osd.8 [v2:172.21.15.113:6816/3569181558,v1:172.21.15.113:6817/3569181558] boot 2024-05-03T18:42:34.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:42:34 smithi045 ceph-mon[33299]: osdmap e53: 9 total, 9 up, 9 in 2024-05-03T18:42:34.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:42:34 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-03T18:42:34.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:42:34 smithi045 ceph-mon[33299]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-05-03T18:42:35.031 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:42:34 smithi032 ceph-mon[29763]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi113", "root=default"]}]': finished 2024-05-03T18:42:35.031 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:42:34 smithi032 ceph-mon[29763]: osdmap e52: 9 total, 8 up, 9 in 2024-05-03T18:42:35.031 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:42:34 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-03T18:42:35.031 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:42:34 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-03T18:42:35.031 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:42:34 smithi032 ceph-mon[29763]: osd.8 [v2:172.21.15.113:6816/3569181558,v1:172.21.15.113:6817/3569181558] boot 2024-05-03T18:42:35.031 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:42:34 smithi032 ceph-mon[29763]: osdmap e53: 9 total, 9 up, 9 in 2024-05-03T18:42:35.031 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:42:34 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-03T18:42:35.031 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:42:34 smithi032 ceph-mon[29763]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-05-03T18:42:35.592 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f500455e-097b-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi032:vg_nvme/lv_1 2024-05-03T18:42:35.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:42:35 smithi032 ceph-mon[29763]: purged_snaps scrub starts 2024-05-03T18:42:35.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:42:35 smithi032 ceph-mon[29763]: purged_snaps scrub ok 2024-05-03T18:42:35.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:42:35 smithi032 ceph-mon[29763]: osdmap e54: 9 total, 9 up, 9 in 2024-05-03T18:42:35.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:42:35 smithi032 ceph-mon[29763]: Detected new or changed devices on smithi113 2024-05-03T18:42:35.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:42:35 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:42:35.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:42:35 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:42:35.819 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:42:35 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:42:35.819 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:42:35 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:42:35.819 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:42:35 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:42:35.819 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:42:35 smithi032 ceph-mon[29763]: Adjusting osd_memory_target on smithi113 to 5696M 2024-05-03T18:42:35.819 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:42:35 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:42:35.819 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:42:35 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:42:35.819 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:42:35 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:42:35.819 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:42:35 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:42:35.840 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:42:35.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:42:35 smithi045 ceph-mon[33299]: purged_snaps scrub starts 2024-05-03T18:42:35.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:42:35 smithi045 ceph-mon[33299]: purged_snaps scrub ok 2024-05-03T18:42:35.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:42:35 smithi045 ceph-mon[33299]: osdmap e54: 9 total, 9 up, 9 in 2024-05-03T18:42:35.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:42:35 smithi045 ceph-mon[33299]: Detected new or changed devices on smithi113 2024-05-03T18:42:35.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:42:35 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:42:35.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:42:35 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:42:35.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:42:35 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:42:35.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:42:35 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:42:35.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:42:35 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:42:35.999 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:42:35 smithi045 ceph-mon[33299]: Adjusting osd_memory_target on smithi113 to 5696M 2024-05-03T18:42:35.999 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:42:35 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:42:35.999 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:42:35 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:42:35.999 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:42:35 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:42:35.999 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:42:35 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:42:36.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:42:35 smithi113 ceph-mon[33540]: purged_snaps scrub starts 2024-05-03T18:42:36.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:42:35 smithi113 ceph-mon[33540]: purged_snaps scrub ok 2024-05-03T18:42:36.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:42:35 smithi113 ceph-mon[33540]: osdmap e54: 9 total, 9 up, 9 in 2024-05-03T18:42:36.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:42:35 smithi113 ceph-mon[33540]: Detected new or changed devices on smithi113 2024-05-03T18:42:36.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:42:35 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:42:36.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:42:35 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:42:36.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:42:35 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:42:36.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:42:35 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:42:36.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:42:35 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:42:36.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:42:35 smithi113 ceph-mon[33540]: Adjusting osd_memory_target on smithi113 to 5696M 2024-05-03T18:42:36.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:42:35 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:42:36.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:42:35 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:42:36.052 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:42:35 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:42:36.052 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:42:35 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:42:36.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:42:36 smithi032 ceph-mon[29763]: osdmap e55: 9 total, 9 up, 9 in 2024-05-03T18:42:36.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:42:36 smithi032 ceph-mon[29763]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-05-03T18:42:36.997 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:42:36 smithi045 ceph-mon[33299]: osdmap e55: 9 total, 9 up, 9 in 2024-05-03T18:42:36.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:42:36 smithi045 ceph-mon[33299]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-05-03T18:42:37.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:42:36 smithi113 ceph-mon[33540]: osdmap e55: 9 total, 9 up, 9 in 2024-05-03T18:42:37.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:42:36 smithi113 ceph-mon[33540]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-05-03T18:42:37.551 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:42:37.549+0000 7f3f5c70d700 1 -- 172.21.15.32:0/3673897794 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f54102420 msgr2=0x7f3f54102840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:42:37.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:42:37.549+0000 7f3f5c70d700 1 --2- 172.21.15.32:0/3673897794 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f54102420 0x7f3f54102840 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f3f500099c0 tx=0x7f3f50009cd0 comp rx=0 tx=0).stop 2024-05-03T18:42:37.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:42:37.549+0000 7f3f5c70d700 1 -- 172.21.15.32:0/3673897794 shutdown_connections 2024-05-03T18:42:37.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:42:37.549+0000 7f3f5c70d700 1 --2- 172.21.15.32:0/3673897794 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3f54103fc0 0x7f3f5410a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:37.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:42:37.549+0000 7f3f5c70d700 1 --2- 172.21.15.32:0/3673897794 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f54102420 0x7f3f54102840 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:37.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:42:37.549+0000 7f3f5c70d700 1 --2- 172.21.15.32:0/3673897794 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f54103620 0x7f3f54103a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:37.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:42:37.549+0000 7f3f5c70d700 1 -- 172.21.15.32:0/3673897794 >> 172.21.15.32:0/3673897794 conn(0x7f3f540fd9c0 msgr2=0x7f3f540ffe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:42:37.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:42:37.549+0000 7f3f5c70d700 1 -- 172.21.15.32:0/3673897794 shutdown_connections 2024-05-03T18:42:37.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:42:37.549+0000 7f3f5c70d700 1 -- 172.21.15.32:0/3673897794 wait complete. 2024-05-03T18:42:37.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:42:37.549+0000 7f3f5c70d700 1 Processor -- start 2024-05-03T18:42:37.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:42:37.549+0000 7f3f5c70d700 1 -- start start 2024-05-03T18:42:37.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:42:37.550+0000 7f3f5c70d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f54102420 0x7f3f54199660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:42:37.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:42:37.550+0000 7f3f5c70d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f54103620 0x7f3f54199ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:42:37.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:42:37.550+0000 7f3f5c70d700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3f54103fc0 0x7f3f541a0c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:42:37.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:42:37.550+0000 7f3f5c70d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f3f5410dec0 con 0x7f3f54103620 2024-05-03T18:42:37.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:42:37.550+0000 7f3f5c70d700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3f5410dd40 con 0x7f3f54102420 2024-05-03T18:42:37.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:42:37.550+0000 7f3f5c70d700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f3f5410e040 con 0x7f3f54103fc0 2024-05-03T18:42:37.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:42:37.550+0000 7f3f59ca8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f54103620 0x7f3f54199ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:42:37.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:42:37.550+0000 7f3f59ca8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f54103620 0x7f3f54199ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38552/0 (socket says 172.21.15.32:38552) 2024-05-03T18:42:37.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:42:37.550+0000 7f3f59ca8700 1 -- 172.21.15.32:0/552561849 learned_addr learned my addr 172.21.15.32:0/552561849 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:42:37.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:42:37.550+0000 7f3f5a4a9700 1 --2- 172.21.15.32:0/552561849 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f54102420 0x7f3f54199660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:42:37.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:42:37.550+0000 7f3f5acaa700 1 --2- 172.21.15.32:0/552561849 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3f54103fc0 0x7f3f541a0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:42:37.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:42:37.550+0000 7f3f59ca8700 1 -- 172.21.15.32:0/552561849 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3f54103fc0 msgr2=0x7f3f541a0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:42:37.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:42:37.550+0000 7f3f59ca8700 1 --2- 172.21.15.32:0/552561849 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3f54103fc0 0x7f3f541a0c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:37.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:42:37.550+0000 7f3f59ca8700 1 -- 172.21.15.32:0/552561849 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f54102420 msgr2=0x7f3f54199660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:42:37.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:42:37.550+0000 7f3f59ca8700 1 --2- 172.21.15.32:0/552561849 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f54102420 0x7f3f54199660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:37.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:42:37.550+0000 7f3f59ca8700 1 -- 172.21.15.32:0/552561849 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f541a12c0 con 0x7f3f54103620 2024-05-03T18:42:37.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:42:37.550+0000 7f3f5a4a9700 1 --2- 172.21.15.32:0/552561849 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f54102420 0x7f3f54199660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T18:42:37.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:42:37.550+0000 7f3f59ca8700 1 --2- 172.21.15.32:0/552561849 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f54103620 0x7f3f54199ba0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f3f4400d980 tx=0x7f3f4400dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:42:37.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:42:37.550+0000 7f3f4b7fe700 1 -- 172.21.15.32:0/552561849 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f44016070 con 0x7f3f54103620 2024-05-03T18:42:37.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:42:37.551+0000 7f3f5c70d700 1 -- 172.21.15.32:0/552561849 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f541a15b0 con 0x7f3f54103620 2024-05-03T18:42:37.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:42:37.551+0000 7f3f4b7fe700 1 -- 172.21.15.32:0/552561849 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f44010460 con 0x7f3f54103620 2024-05-03T18:42:37.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:42:37.551+0000 7f3f4b7fe700 1 -- 172.21.15.32:0/552561849 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f4400f620 con 0x7f3f54103620 2024-05-03T18:42:37.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:42:37.551+0000 7f3f5c70d700 1 -- 172.21.15.32:0/552561849 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f541a1b40 con 0x7f3f54103620 2024-05-03T18:42:37.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:42:37.552+0000 7f3f5c70d700 1 -- 172.21.15.32:0/552561849 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f5406a190 con 0x7f3f54103620 2024-05-03T18:42:37.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:42:37.553+0000 7f3f4b7fe700 1 -- 172.21.15.32:0/552561849 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f3f44019070 con 0x7f3f54103620 2024-05-03T18:42:37.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:42:37.554+0000 7f3f4b7fe700 1 --2- 172.21.15.32:0/552561849 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f3f400b2880 0x7f3f400b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:42:37.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:42:37.554+0000 7f3f5a4a9700 1 --2- 172.21.15.32:0/552561849 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f3f400b2880 0x7f3f400b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:42:37.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:42:37.554+0000 7f3f4b7fe700 1 -- 172.21.15.32:0/552561849 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 5829+0+0 (secure 0 0 0) 0x7f3f440e14f0 con 0x7f3f54103620 2024-05-03T18:42:37.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:42:37.554+0000 7f3f5a4a9700 1 --2- 172.21.15.32:0/552561849 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f3f400b2880 0x7f3f400b4d40 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f3f5000b580 tx=0x7f3f50005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:42:37.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:42:37.557+0000 7f3f4b7fe700 1 -- 172.21.15.32:0/552561849 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f3f440a9b20 con 0x7f3f54103620 2024-05-03T18:42:37.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:42:37.784+0000 7f3f5c70d700 1 -- 172.21.15.32:0/552561849 --> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi032:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f3f540ffb90 con 0x7f3f400b2880 2024-05-03T18:42:38.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:42:37 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T18:42:38.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:42:37 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T18:42:38.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:42:37 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:42:38.247 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:42:37 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T18:42:38.248 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:42:37 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T18:42:38.248 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:42:37 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:42:38.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:42:37 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T18:42:38.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:42:37 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T18:42:38.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:42:37 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:42:39.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:42:38 smithi032 ceph-mon[29763]: from='client.14433 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi032:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:42:39.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:42:38 smithi032 ceph-mon[29763]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 135 KiB/s, 0 objects/s recovering 2024-05-03T18:42:39.247 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:42:38 smithi045 ceph-mon[33299]: from='client.14433 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi032:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:42:39.248 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:42:38 smithi045 ceph-mon[33299]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 135 KiB/s, 0 objects/s recovering 2024-05-03T18:42:39.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:42:38 smithi113 ceph-mon[33540]: from='client.14433 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi032:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:42:39.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:42:38 smithi113 ceph-mon[33540]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 135 KiB/s, 0 objects/s recovering 2024-05-03T18:42:41.247 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:42:40 smithi045 ceph-mon[33299]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-05-03T18:42:41.248 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:42:40 smithi045 ceph-mon[33299]: from='client.? 172.21.15.32:0/1544677152' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1d35c435-36b8-40e8-a2d4-0efe904f1107"}]: dispatch 2024-05-03T18:42:41.248 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:42:40 smithi045 ceph-mon[33299]: from='client.? 172.21.15.32:0/1544677152' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1d35c435-36b8-40e8-a2d4-0efe904f1107"}]': finished 2024-05-03T18:42:41.248 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:42:40 smithi045 ceph-mon[33299]: osdmap e56: 10 total, 9 up, 10 in 2024-05-03T18:42:41.248 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:42:40 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-03T18:42:41.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:42:40 smithi113 ceph-mon[33540]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-05-03T18:42:41.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:42:40 smithi113 ceph-mon[33540]: from='client.? 172.21.15.32:0/1544677152' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1d35c435-36b8-40e8-a2d4-0efe904f1107"}]: dispatch 2024-05-03T18:42:41.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:42:40 smithi113 ceph-mon[33540]: from='client.? 172.21.15.32:0/1544677152' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1d35c435-36b8-40e8-a2d4-0efe904f1107"}]': finished 2024-05-03T18:42:41.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:42:40 smithi113 ceph-mon[33540]: osdmap e56: 10 total, 9 up, 10 in 2024-05-03T18:42:41.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:42:40 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-03T18:42:41.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:42:40 smithi032 ceph-mon[29763]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-05-03T18:42:41.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:42:40 smithi032 ceph-mon[29763]: from='client.? 172.21.15.32:0/1544677152' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1d35c435-36b8-40e8-a2d4-0efe904f1107"}]: dispatch 2024-05-03T18:42:41.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:42:40 smithi032 ceph-mon[29763]: from='client.? 172.21.15.32:0/1544677152' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1d35c435-36b8-40e8-a2d4-0efe904f1107"}]': finished 2024-05-03T18:42:41.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:42:40 smithi032 ceph-mon[29763]: osdmap e56: 10 total, 9 up, 10 in 2024-05-03T18:42:41.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:42:40 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-03T18:42:42.247 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:42:41 smithi045 ceph-mon[33299]: from='client.? 172.21.15.32:0/3422504210' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T18:42:42.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:42:41 smithi113 ceph-mon[33540]: from='client.? 172.21.15.32:0/3422504210' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T18:42:42.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:42:41 smithi032 ceph-mon[29763]: from='client.? 172.21.15.32:0/3422504210' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T18:42:43.247 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:42:42 smithi045 ceph-mon[33299]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-05-03T18:42:43.300 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:42:42 smithi113 ceph-mon[33540]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-05-03T18:42:43.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:42:42 smithi032 ceph-mon[29763]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-05-03T18:42:45.135 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:42:44 smithi032 ceph-mon[29763]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-05-03T18:42:45.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:42:44 smithi113 ceph-mon[33540]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-05-03T18:42:45.497 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:42:44 smithi045 ceph-mon[33299]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-05-03T18:42:47.250 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:42:47 smithi032 ceph-mon[29763]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-03T18:42:47.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:42:47 smithi113 ceph-mon[33540]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-03T18:42:47.497 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:42:47 smithi045 ceph-mon[33299]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-03T18:42:49.293 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:42:49 smithi032 ceph-mon[29763]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-05-03T18:42:49.293 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:42:49 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-03T18:42:49.293 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:42:49 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:42:49.293 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:42:49 smithi032 ceph-mon[29763]: Deploying daemon osd.9 on smithi032 2024-05-03T18:42:49.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:42:49 smithi113 ceph-mon[33540]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-05-03T18:42:49.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:42:49 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-03T18:42:49.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:42:49 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:42:49.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:42:49 smithi113 ceph-mon[33540]: Deploying daemon osd.9 on smithi032 2024-05-03T18:42:49.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:42:49 smithi045 ceph-mon[33299]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-05-03T18:42:49.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:42:49 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-03T18:42:49.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:42:49 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:42:49.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:42:49 smithi045 ceph-mon[33299]: Deploying daemon osd.9 on smithi032 2024-05-03T18:42:51.204 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:42:51 smithi032 ceph-mon[29763]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-05-03T18:42:51.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:42:51 smithi113 ceph-mon[33540]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-05-03T18:42:51.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:42:51 smithi045 ceph-mon[33299]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-05-03T18:42:53.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:42:53 smithi113 ceph-mon[33540]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-05-03T18:42:53.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:42:53 smithi032 ceph-mon[29763]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-05-03T18:42:53.497 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:42:53 smithi045 ceph-mon[33299]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-05-03T18:42:55.045 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:42:55 smithi032 ceph-mon[29763]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-05-03T18:42:55.046 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:42:55 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:42:55.046 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:42:55 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:42:55.046 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:42:55 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:42:55.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:42:55 smithi113 ceph-mon[33540]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-05-03T18:42:55.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:42:55 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:42:55.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:42:55 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:42:55.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:42:55 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:42:55.497 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:42:55 smithi045 ceph-mon[33299]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-05-03T18:42:55.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:42:55 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:42:55.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:42:55 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:42:55.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:42:55 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:42:57.183 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:42:57 smithi032 ceph-mon[29763]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-05-03T18:42:57.300 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:42:57 smithi045 ceph-mon[33299]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-05-03T18:42:57.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:42:57 smithi113 ceph-mon[33540]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-05-03T18:42:58.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:42:58.293+0000 7f3f4b7fe700 1 -- 172.21.15.32:0/552561849 <== mgr.14152 v2:172.21.15.32:6800/1796547904 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f3f540ffb90 con 0x7f3f400b2880 2024-05-03T18:42:58.294 INFO:teuthology.orchestra.run.smithi032.stdout:Created osd(s) 9 on host 'smithi032' 2024-05-03T18:42:58.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:42:58.295+0000 7f3f5c70d700 1 -- 172.21.15.32:0/552561849 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f3f400b2880 msgr2=0x7f3f400b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:42:58.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:42:58.295+0000 7f3f5c70d700 1 --2- 172.21.15.32:0/552561849 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f3f400b2880 0x7f3f400b4d40 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f3f5000b580 tx=0x7f3f50005fb0 comp rx=0 tx=0).stop 2024-05-03T18:42:58.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:42:58.295+0000 7f3f5c70d700 1 -- 172.21.15.32:0/552561849 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f54103620 msgr2=0x7f3f54199ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:42:58.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:42:58.295+0000 7f3f5c70d700 1 --2- 172.21.15.32:0/552561849 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f54103620 0x7f3f54199ba0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f3f4400d980 tx=0x7f3f4400dd40 comp rx=0 tx=0).stop 2024-05-03T18:42:58.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:42:58.295+0000 7f3f5c70d700 1 -- 172.21.15.32:0/552561849 shutdown_connections 2024-05-03T18:42:58.296 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:42:58.295+0000 7f3f5c70d700 1 --2- 172.21.15.32:0/552561849 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f3f400b2880 0x7f3f400b4d40 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:58.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:42:58.295+0000 7f3f5c70d700 1 --2- 172.21.15.32:0/552561849 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3f54103fc0 0x7f3f541a0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:58.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:42:58.295+0000 7f3f5c70d700 1 --2- 172.21.15.32:0/552561849 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f54102420 0x7f3f54199660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:58.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:42:58.295+0000 7f3f5c70d700 1 --2- 172.21.15.32:0/552561849 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f54103620 0x7f3f54199ba0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:42:58.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:42:58.295+0000 7f3f5c70d700 1 -- 172.21.15.32:0/552561849 >> 172.21.15.32:0/552561849 conn(0x7f3f540fd9c0 msgr2=0x7f3f540ff480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:42:58.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:42:58.295+0000 7f3f5c70d700 1 -- 172.21.15.32:0/552561849 shutdown_connections 2024-05-03T18:42:58.297 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:42:58.295+0000 7f3f5c70d700 1 -- 172.21.15.32:0/552561849 wait complete. 2024-05-03T18:42:58.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:42:58 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:42:58.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:42:58 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:42:58.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:42:58 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:42:58.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:42:58 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:42:58.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:42:58 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:42:58.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:42:58 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:42:58.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:42:58 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:42:58.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:42:58 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:42:58.569 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:42:58 smithi032 ceph-mon[29763]: from='osd.9 [v2:172.21.15.32:6826/4290196345,v1:172.21.15.32:6827/4290196345]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-03T18:42:58.729 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:42:58 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:42:58.730 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:42:58 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:42:58.730 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:42:58 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:42:58.730 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:42:58 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:42:58.730 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:42:58 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:42:58.730 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:42:58 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:42:58.730 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:42:58 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:42:58.730 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:42:58 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:42:58.730 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:42:58 smithi045 ceph-mon[33299]: from='osd.9 [v2:172.21.15.32:6826/4290196345,v1:172.21.15.32:6827/4290196345]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-03T18:42:58.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:42:58 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:42:58.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:42:58 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:42:58.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:42:58 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:42:58.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:42:58 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:42:58.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:42:58 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:42:58.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:42:58 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:42:58.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:42:58 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:42:58.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:42:58 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:42:58.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:42:58 smithi113 ceph-mon[33540]: from='osd.9 [v2:172.21.15.32:6826/4290196345,v1:172.21.15.32:6827/4290196345]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-03T18:42:58.912 DEBUG:teuthology.orchestra.run.smithi032:osd.9> sudo journalctl -f -n 0 -u ceph-f500455e-097b-11ef-bc97-c7b262605968@osd.9.service 2024-05-03T18:42:58.914 INFO:tasks.cephadm:Deploying osd.10 on smithi045 with /dev/vg_nvme/lv_1... 2024-05-03T18:42:58.914 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f500455e-097b-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-03T18:42:59.157 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/mon.b/config 2024-05-03T18:42:59.559 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:42:59 smithi045 ceph-mon[33299]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-05-03T18:42:59.559 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:42:59 smithi045 ceph-mon[33299]: from='osd.9 [v2:172.21.15.32:6826/4290196345,v1:172.21.15.32:6827/4290196345]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-03T18:42:59.559 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:42:59 smithi045 ceph-mon[33299]: osdmap e57: 10 total, 9 up, 10 in 2024-05-03T18:42:59.559 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:42:59 smithi045 ceph-mon[33299]: from='osd.9 [v2:172.21.15.32:6826/4290196345,v1:172.21.15.32:6827/4290196345]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi032", "root=default"]}]: dispatch 2024-05-03T18:42:59.559 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:42:59 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-03T18:42:59.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:42:59 smithi032 ceph-mon[29763]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-05-03T18:42:59.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:42:59 smithi032 ceph-mon[29763]: from='osd.9 [v2:172.21.15.32:6826/4290196345,v1:172.21.15.32:6827/4290196345]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-03T18:42:59.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:42:59 smithi032 ceph-mon[29763]: osdmap e57: 10 total, 9 up, 10 in 2024-05-03T18:42:59.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:42:59 smithi032 ceph-mon[29763]: from='osd.9 [v2:172.21.15.32:6826/4290196345,v1:172.21.15.32:6827/4290196345]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi032", "root=default"]}]: dispatch 2024-05-03T18:42:59.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:42:59 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-03T18:42:59.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:42:59 smithi113 ceph-mon[33540]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-05-03T18:42:59.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:42:59 smithi113 ceph-mon[33540]: from='osd.9 [v2:172.21.15.32:6826/4290196345,v1:172.21.15.32:6827/4290196345]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-03T18:42:59.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:42:59 smithi113 ceph-mon[33540]: osdmap e57: 10 total, 9 up, 10 in 2024-05-03T18:42:59.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:42:59 smithi113 ceph-mon[33540]: from='osd.9 [v2:172.21.15.32:6826/4290196345,v1:172.21.15.32:6827/4290196345]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi032", "root=default"]}]: dispatch 2024-05-03T18:42:59.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:42:59 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-03T18:43:00.365 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 18:43:00 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T18:43:00.261+0000 7fdaaac3a700 -1 osd.9 0 waiting for initial osdmap 2024-05-03T18:43:00.365 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 18:43:00 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T18:43:00.273+0000 7fdaa6a2d700 -1 osd.9 58 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-03T18:43:01.171 INFO:teuthology.orchestra.run.smithi045.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-03T18:43:01.171 INFO:teuthology.orchestra.run.smithi045.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-03T18:43:01.171 INFO:teuthology.orchestra.run.smithi045.stderr: stderr: 10+0 records in 2024-05-03T18:43:01.171 INFO:teuthology.orchestra.run.smithi045.stderr:10+0 records out 2024-05-03T18:43:01.171 INFO:teuthology.orchestra.run.smithi045.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00840333 s, 1.2 GB/s 2024-05-03T18:43:01.171 INFO:teuthology.orchestra.run.smithi045.stderr:--> Zapping successful for: 2024-05-03T18:43:01.495 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:01 smithi032 ceph-mon[29763]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-05-03T18:43:01.495 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:01 smithi032 ceph-mon[29763]: from='osd.9 [v2:172.21.15.32:6826/4290196345,v1:172.21.15.32:6827/4290196345]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi032", "root=default"]}]': finished 2024-05-03T18:43:01.495 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:01 smithi032 ceph-mon[29763]: osdmap e58: 10 total, 9 up, 10 in 2024-05-03T18:43:01.495 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:01 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-03T18:43:01.495 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:01 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-03T18:43:01.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:01 smithi113 ceph-mon[33540]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-05-03T18:43:01.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:01 smithi113 ceph-mon[33540]: from='osd.9 [v2:172.21.15.32:6826/4290196345,v1:172.21.15.32:6827/4290196345]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi032", "root=default"]}]': finished 2024-05-03T18:43:01.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:01 smithi113 ceph-mon[33540]: osdmap e58: 10 total, 9 up, 10 in 2024-05-03T18:43:01.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:01 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-03T18:43:01.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:01 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-03T18:43:01.653 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:01 smithi045 ceph-mon[33299]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-05-03T18:43:01.653 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:01 smithi045 ceph-mon[33299]: from='osd.9 [v2:172.21.15.32:6826/4290196345,v1:172.21.15.32:6827/4290196345]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi032", "root=default"]}]': finished 2024-05-03T18:43:01.653 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:01 smithi045 ceph-mon[33299]: osdmap e58: 10 total, 9 up, 10 in 2024-05-03T18:43:01.653 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:01 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-03T18:43:01.653 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:01 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-03T18:43:01.945 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f500455e-097b-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi045:vg_nvme/lv_1 2024-05-03T18:43:02.201 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/mon.b/config 2024-05-03T18:43:02.530 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:02 smithi045 ceph-mon[33299]: purged_snaps scrub starts 2024-05-03T18:43:02.530 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:02 smithi045 ceph-mon[33299]: purged_snaps scrub ok 2024-05-03T18:43:02.530 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:02 smithi045 ceph-mon[33299]: osd.9 [v2:172.21.15.32:6826/4290196345,v1:172.21.15.32:6827/4290196345] boot 2024-05-03T18:43:02.530 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:02 smithi045 ceph-mon[33299]: osdmap e59: 10 total, 10 up, 10 in 2024-05-03T18:43:02.530 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:02 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-03T18:43:02.530 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:02 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:43:02.530 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:02 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:43:02.530 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:02 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:43:02.531 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:02 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:43:02.531 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:02 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:43:02.531 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:02 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:43:02.531 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:02 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:43:02.531 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:02 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:43:02.531 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:02 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:43:02.531 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:02 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:43:02.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:02 smithi113 ceph-mon[33540]: purged_snaps scrub starts 2024-05-03T18:43:02.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:02 smithi113 ceph-mon[33540]: purged_snaps scrub ok 2024-05-03T18:43:02.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:02 smithi113 ceph-mon[33540]: osd.9 [v2:172.21.15.32:6826/4290196345,v1:172.21.15.32:6827/4290196345] boot 2024-05-03T18:43:02.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:02 smithi113 ceph-mon[33540]: osdmap e59: 10 total, 10 up, 10 in 2024-05-03T18:43:02.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:02 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-03T18:43:02.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:02 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:43:02.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:02 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:43:02.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:02 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:43:02.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:02 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:43:02.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:02 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:43:02.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:02 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:43:02.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:02 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:43:02.552 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:02 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:43:02.552 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:02 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:43:02.552 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:02 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:43:02.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:02 smithi032 ceph-mon[29763]: purged_snaps scrub starts 2024-05-03T18:43:02.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:02 smithi032 ceph-mon[29763]: purged_snaps scrub ok 2024-05-03T18:43:02.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:02 smithi032 ceph-mon[29763]: osd.9 [v2:172.21.15.32:6826/4290196345,v1:172.21.15.32:6827/4290196345] boot 2024-05-03T18:43:02.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:02 smithi032 ceph-mon[29763]: osdmap e59: 10 total, 10 up, 10 in 2024-05-03T18:43:02.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:02 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-03T18:43:02.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:02 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:43:02.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:02 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:43:02.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:02 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:43:02.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:02 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:43:02.569 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:02 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:43:02.569 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:02 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:43:02.569 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:02 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:43:02.569 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:02 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:43:02.569 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:02 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:43:02.569 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:02 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:43:03.402 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:03 smithi045 ceph-mon[33299]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-05-03T18:43:03.402 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:03 smithi045 ceph-mon[33299]: Detected new or changed devices on smithi032 2024-05-03T18:43:03.402 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:03 smithi045 ceph-mon[33299]: Adjusting osd_memory_target on smithi032 to 4273M 2024-05-03T18:43:03.402 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:03 smithi045 ceph-mon[33299]: osdmap e60: 10 total, 10 up, 10 in 2024-05-03T18:43:03.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:03 smithi113 ceph-mon[33540]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-05-03T18:43:03.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:03 smithi113 ceph-mon[33540]: Detected new or changed devices on smithi032 2024-05-03T18:43:03.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:03 smithi113 ceph-mon[33540]: Adjusting osd_memory_target on smithi032 to 4273M 2024-05-03T18:43:03.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:03 smithi113 ceph-mon[33540]: osdmap e60: 10 total, 10 up, 10 in 2024-05-03T18:43:03.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:03 smithi032 ceph-mon[29763]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-05-03T18:43:03.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:03 smithi032 ceph-mon[29763]: Detected new or changed devices on smithi032 2024-05-03T18:43:03.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:03 smithi032 ceph-mon[29763]: Adjusting osd_memory_target on smithi032 to 4273M 2024-05-03T18:43:03.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:03 smithi032 ceph-mon[29763]: osdmap e60: 10 total, 10 up, 10 in 2024-05-03T18:43:03.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:43:03.873+0000 7efcde534700 1 -- 172.21.15.45:0/3750053287 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efcd8103620 msgr2=0x7efcd8103a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:43:03.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:43:03.873+0000 7efcde534700 1 --2- 172.21.15.45:0/3750053287 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efcd8103620 0x7efcd8103a80 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7efccc0099c0 tx=0x7efccc009cd0 comp rx=0 tx=0).stop 2024-05-03T18:43:03.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:43:03.874+0000 7efcde534700 1 -- 172.21.15.45:0/3750053287 shutdown_connections 2024-05-03T18:43:03.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:43:03.874+0000 7efcde534700 1 --2- 172.21.15.45:0/3750053287 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efcd8103fc0 0x7efcd810a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:03.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:43:03.874+0000 7efcde534700 1 --2- 172.21.15.45:0/3750053287 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efcd8102420 0x7efcd8102840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:03.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:43:03.874+0000 7efcde534700 1 --2- 172.21.15.45:0/3750053287 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efcd8103620 0x7efcd8103a80 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:03.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:43:03.874+0000 7efcde534700 1 -- 172.21.15.45:0/3750053287 >> 172.21.15.45:0/3750053287 conn(0x7efcd80fd9c0 msgr2=0x7efcd80ffe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:43:03.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:43:03.874+0000 7efcde534700 1 -- 172.21.15.45:0/3750053287 shutdown_connections 2024-05-03T18:43:03.876 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:43:03.874+0000 7efcde534700 1 -- 172.21.15.45:0/3750053287 wait complete. 2024-05-03T18:43:03.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:43:03.874+0000 7efcde534700 1 Processor -- start 2024-05-03T18:43:03.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:43:03.874+0000 7efcde534700 1 -- start start 2024-05-03T18:43:03.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:43:03.875+0000 7efcde534700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efcd8102420 0x7efcd8199670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:43:03.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:43:03.875+0000 7efcde534700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efcd8103620 0x7efcd8199bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:43:03.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:43:03.875+0000 7efcde534700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efcd8103fc0 0x7efcd81a0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:43:03.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:43:03.875+0000 7efcde534700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7efcd810dea0 con 0x7efcd8103620 2024-05-03T18:43:03.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:43:03.875+0000 7efcde534700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7efcd810dd20 con 0x7efcd8102420 2024-05-03T18:43:03.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:43:03.875+0000 7efcde534700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7efcd810e020 con 0x7efcd8103fc0 2024-05-03T18:43:03.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:43:03.875+0000 7efcd7fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efcd8102420 0x7efcd8199670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:43:03.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:43:03.875+0000 7efcd7fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efcd8102420 0x7efcd8199670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35382/0 (socket says 172.21.15.45:35382) 2024-05-03T18:43:03.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:43:03.875+0000 7efcd7fff700 1 -- 172.21.15.45:0/177152919 learned_addr learned my addr 172.21.15.45:0/177152919 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-05-03T18:43:03.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:43:03.875+0000 7efcd77fe700 1 --2- 172.21.15.45:0/177152919 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efcd8103620 0x7efcd8199bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:43:03.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:43:03.875+0000 7efcdcad1700 1 --2- 172.21.15.45:0/177152919 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efcd8103fc0 0x7efcd81a0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:43:03.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:43:03.875+0000 7efcd7fff700 1 -- 172.21.15.45:0/177152919 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efcd8103fc0 msgr2=0x7efcd81a0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:43:03.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:43:03.875+0000 7efcd7fff700 1 --2- 172.21.15.45:0/177152919 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efcd8103fc0 0x7efcd81a0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:03.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:43:03.875+0000 7efcd7fff700 1 -- 172.21.15.45:0/177152919 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efcd8103620 msgr2=0x7efcd8199bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:43:03.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:43:03.875+0000 7efcd7fff700 1 --2- 172.21.15.45:0/177152919 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efcd8103620 0x7efcd8199bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:03.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:43:03.875+0000 7efcd7fff700 1 -- 172.21.15.45:0/177152919 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcd81a1230 con 0x7efcd8102420 2024-05-03T18:43:03.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:43:03.875+0000 7efcdcad1700 1 --2- 172.21.15.45:0/177152919 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efcd8103fc0 0x7efcd81a0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:43:03.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:43:03.875+0000 7efcd7fff700 1 --2- 172.21.15.45:0/177152919 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efcd8102420 0x7efcd8199670 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7efcc800d980 tx=0x7efcc800dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:43:03.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:43:03.875+0000 7efcd57fa700 1 -- 172.21.15.45:0/177152919 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcc8017030 con 0x7efcd8102420 2024-05-03T18:43:03.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:43:03.876+0000 7efcd57fa700 1 -- 172.21.15.45:0/177152919 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efcc80044d0 con 0x7efcd8102420 2024-05-03T18:43:03.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:43:03.876+0000 7efcde534700 1 -- 172.21.15.45:0/177152919 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcd81a1520 con 0x7efcd8102420 2024-05-03T18:43:03.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:43:03.876+0000 7efcd57fa700 1 -- 172.21.15.45:0/177152919 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcc80039e0 con 0x7efcd8102420 2024-05-03T18:43:03.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:43:03.876+0000 7efcde534700 1 -- 172.21.15.45:0/177152919 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcd81a1a60 con 0x7efcd8102420 2024-05-03T18:43:03.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:43:03.876+0000 7efcde534700 1 -- 172.21.15.45:0/177152919 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efcd806a190 con 0x7efcd8102420 2024-05-03T18:43:03.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:43:03.879+0000 7efcd57fa700 1 -- 172.21.15.45:0/177152919 <== mon.2 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7efcc801a070 con 0x7efcd8102420 2024-05-03T18:43:03.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:43:03.879+0000 7efcd57fa700 1 --2- 172.21.15.45:0/177152919 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7efcc00b2880 0x7efcc00b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:43:03.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:43:03.879+0000 7efcd57fa700 1 -- 172.21.15.45:0/177152919 <== mon.2 v2:172.21.15.45:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 6261+0+0 (secure 0 0 0) 0x7efcc80e1710 con 0x7efcd8102420 2024-05-03T18:43:03.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:43:03.879+0000 7efcd77fe700 1 --2- 172.21.15.45:0/177152919 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7efcc00b2880 0x7efcc00b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:43:03.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:43:03.880+0000 7efcd77fe700 1 --2- 172.21.15.45:0/177152919 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7efcc00b2880 0x7efcc00b4d40 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7efccc009990 tx=0x7efccc0096b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:43:03.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:43:03.883+0000 7efcd57fa700 1 -- 172.21.15.45:0/177152919 <== mon.2 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7efcc80a9c30 con 0x7efcd8102420 2024-05-03T18:43:04.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:43:04.106+0000 7efcde534700 1 -- 172.21.15.45:0/177152919 --> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7efcd80ffb90 con 0x7efcc00b2880 2024-05-03T18:43:04.497 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:04 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T18:43:04.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:04 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T18:43:04.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:04 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:43:04.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:04 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T18:43:04.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:04 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T18:43:04.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:04 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:43:04.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:04 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T18:43:04.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:04 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T18:43:04.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:04 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:43:05.497 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:05 smithi045 ceph-mon[33299]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-03T18:43:05.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:05 smithi045 ceph-mon[33299]: from='client.24311 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:43:05.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:05 smithi113 ceph-mon[33540]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-03T18:43:05.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:05 smithi113 ceph-mon[33540]: from='client.24311 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:43:05.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:05 smithi032 ceph-mon[29763]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-03T18:43:05.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:05 smithi032 ceph-mon[29763]: from='client.24311 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:43:07.497 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:07 smithi045 ceph-mon[33299]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-03T18:43:07.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:07 smithi045 ceph-mon[33299]: from='client.? 172.21.15.45:0/735244395' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9af7879e-aa8d-4ef8-b289-4b145b614820"}]: dispatch 2024-05-03T18:43:07.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:07 smithi045 ceph-mon[33299]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9af7879e-aa8d-4ef8-b289-4b145b614820"}]: dispatch 2024-05-03T18:43:07.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:07 smithi045 ceph-mon[33299]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9af7879e-aa8d-4ef8-b289-4b145b614820"}]': finished 2024-05-03T18:43:07.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:07 smithi045 ceph-mon[33299]: osdmap e61: 11 total, 10 up, 11 in 2024-05-03T18:43:07.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:07 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-03T18:43:07.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:07 smithi113 ceph-mon[33540]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-03T18:43:07.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:07 smithi113 ceph-mon[33540]: from='client.? 172.21.15.45:0/735244395' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9af7879e-aa8d-4ef8-b289-4b145b614820"}]: dispatch 2024-05-03T18:43:07.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:07 smithi113 ceph-mon[33540]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9af7879e-aa8d-4ef8-b289-4b145b614820"}]: dispatch 2024-05-03T18:43:07.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:07 smithi113 ceph-mon[33540]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9af7879e-aa8d-4ef8-b289-4b145b614820"}]': finished 2024-05-03T18:43:07.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:07 smithi113 ceph-mon[33540]: osdmap e61: 11 total, 10 up, 11 in 2024-05-03T18:43:07.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:07 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-03T18:43:07.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:07 smithi032 ceph-mon[29763]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-03T18:43:07.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:07 smithi032 ceph-mon[29763]: from='client.? 172.21.15.45:0/735244395' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9af7879e-aa8d-4ef8-b289-4b145b614820"}]: dispatch 2024-05-03T18:43:07.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:07 smithi032 ceph-mon[29763]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9af7879e-aa8d-4ef8-b289-4b145b614820"}]: dispatch 2024-05-03T18:43:07.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:07 smithi032 ceph-mon[29763]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9af7879e-aa8d-4ef8-b289-4b145b614820"}]': finished 2024-05-03T18:43:07.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:07 smithi032 ceph-mon[29763]: osdmap e61: 11 total, 10 up, 11 in 2024-05-03T18:43:07.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:07 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-03T18:43:08.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:08 smithi113 ceph-mon[33540]: from='client.? 172.21.15.45:0/1534221971' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T18:43:08.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:08 smithi032 ceph-mon[29763]: from='client.? 172.21.15.45:0/1534221971' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T18:43:08.747 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:08 smithi045 ceph-mon[33299]: from='client.? 172.21.15.45:0/1534221971' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T18:43:09.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:09 smithi113 ceph-mon[33540]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-03T18:43:09.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:09 smithi032 ceph-mon[29763]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-03T18:43:09.747 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:09 smithi045 ceph-mon[33299]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-03T18:43:11.497 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:11 smithi045 ceph-mon[33299]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-03T18:43:11.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:11 smithi113 ceph-mon[33540]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-03T18:43:11.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:11 smithi032 ceph-mon[29763]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-03T18:43:13.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:13 smithi113 ceph-mon[33540]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-03T18:43:13.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:13 smithi032 ceph-mon[29763]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-03T18:43:13.747 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:13 smithi045 ceph-mon[33299]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-03T18:43:14.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:14 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-03T18:43:14.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:14 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:43:14.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:14 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-03T18:43:14.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:14 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:43:14.741 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:14 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-03T18:43:14.741 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:14 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:43:15.404 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:15 smithi045 ceph-mon[33299]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-03T18:43:15.405 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:15 smithi045 ceph-mon[33299]: Deploying daemon osd.10 on smithi045 2024-05-03T18:43:15.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:15 smithi113 ceph-mon[33540]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-03T18:43:15.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:15 smithi113 ceph-mon[33540]: Deploying daemon osd.10 on smithi045 2024-05-03T18:43:15.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:15 smithi032 ceph-mon[29763]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-03T18:43:15.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:15 smithi032 ceph-mon[29763]: Deploying daemon osd.10 on smithi045 2024-05-03T18:43:17.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:17 smithi113 ceph-mon[33540]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-03T18:43:17.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:17 smithi032 ceph-mon[29763]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-03T18:43:17.708 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:17 smithi045 ceph-mon[33299]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-03T18:43:19.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:19 smithi032 ceph-mon[29763]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-03T18:43:19.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:19 smithi045 ceph-mon[33299]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-03T18:43:19.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:19 smithi113 ceph-mon[33540]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-03T18:43:20.860 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:20 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:43:20.860 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:20 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:43:20.860 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:20 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:43:20.861 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:20 smithi045 ceph-mon[33299]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-03T18:43:21.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:20 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:43:21.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:20 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:43:21.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:20 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:43:21.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:20 smithi113 ceph-mon[33540]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-03T18:43:21.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:20 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:43:21.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:20 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:43:21.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:20 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:43:21.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:20 smithi032 ceph-mon[29763]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-03T18:43:23.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:43:23.323+0000 7efcd57fa700 1 -- 172.21.15.45:0/177152919 <== mgr.14152 v2:172.21.15.32:6800/1796547904 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7efcd80ffb90 con 0x7efcc00b2880 2024-05-03T18:43:23.324 INFO:teuthology.orchestra.run.smithi045.stdout:Created osd(s) 10 on host 'smithi045' 2024-05-03T18:43:23.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:43:23.325+0000 7efcde534700 1 -- 172.21.15.45:0/177152919 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7efcc00b2880 msgr2=0x7efcc00b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:43:23.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:43:23.325+0000 7efcde534700 1 --2- 172.21.15.45:0/177152919 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7efcc00b2880 0x7efcc00b4d40 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7efccc009990 tx=0x7efccc0096b0 comp rx=0 tx=0).stop 2024-05-03T18:43:23.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:43:23.325+0000 7efcde534700 1 -- 172.21.15.45:0/177152919 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efcd8102420 msgr2=0x7efcd8199670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:43:23.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:43:23.325+0000 7efcde534700 1 --2- 172.21.15.45:0/177152919 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efcd8102420 0x7efcd8199670 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7efcc800d980 tx=0x7efcc800dc90 comp rx=0 tx=0).stop 2024-05-03T18:43:23.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:43:23.325+0000 7efcde534700 1 -- 172.21.15.45:0/177152919 shutdown_connections 2024-05-03T18:43:23.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:43:23.325+0000 7efcde534700 1 --2- 172.21.15.45:0/177152919 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7efcc00b2880 0x7efcc00b4d40 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:23.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:43:23.325+0000 7efcde534700 1 --2- 172.21.15.45:0/177152919 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efcd8103fc0 0x7efcd81a0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:23.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:43:23.325+0000 7efcde534700 1 --2- 172.21.15.45:0/177152919 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efcd8102420 0x7efcd8199670 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:23.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:43:23.325+0000 7efcde534700 1 --2- 172.21.15.45:0/177152919 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efcd8103620 0x7efcd8199bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:23.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:43:23.325+0000 7efcde534700 1 -- 172.21.15.45:0/177152919 >> 172.21.15.45:0/177152919 conn(0x7efcd80fd9c0 msgr2=0x7efcd80ff480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:43:23.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:43:23.326+0000 7efcde534700 1 -- 172.21.15.45:0/177152919 shutdown_connections 2024-05-03T18:43:23.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-05-03T18:43:23.326+0000 7efcde534700 1 -- 172.21.15.45:0/177152919 wait complete. 2024-05-03T18:43:23.687 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:23 smithi045 ceph-mon[33299]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-03T18:43:23.687 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:23 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:43:23.687 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:23 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:43:23.688 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:23 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:43:23.688 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:23 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:43:23.688 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:23 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:43:23.688 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:23 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:43:23.688 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:23 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:43:23.688 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:23 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:43:23.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:23 smithi113 ceph-mon[33540]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-03T18:43:23.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:23 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:43:23.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:23 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:43:23.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:23 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:43:23.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:23 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:43:23.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:23 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:43:23.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:23 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:43:23.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:23 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:43:23.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:23 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:43:23.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:23 smithi032 ceph-mon[29763]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-03T18:43:23.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:23 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:43:23.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:23 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:43:23.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:23 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:43:23.819 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:23 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:43:23.819 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:23 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:43:23.819 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:23 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:43:23.819 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:23 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:43:23.819 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:23 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:43:23.957 DEBUG:teuthology.orchestra.run.smithi045:osd.10> sudo journalctl -f -n 0 -u ceph-f500455e-097b-11ef-bc97-c7b262605968@osd.10.service 2024-05-03T18:43:23.959 INFO:tasks.cephadm:Deploying osd.11 on smithi113 with /dev/vg_nvme/lv_1... 2024-05-03T18:43:23.959 DEBUG:teuthology.orchestra.run.smithi113:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f500455e-097b-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-03T18:43:24.204 INFO:teuthology.orchestra.run.smithi113.stderr:Inferring config /var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/mon.c/config 2024-05-03T18:43:24.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:24 smithi045 ceph-mon[33299]: from='osd.10 [v2:172.21.15.45:6824/1817064361,v1:172.21.15.45:6825/1817064361]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-03T18:43:24.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:24 smithi045 ceph-mon[33299]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-03T18:43:24.576 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:24 smithi113 ceph-mon[33540]: from='osd.10 [v2:172.21.15.45:6824/1817064361,v1:172.21.15.45:6825/1817064361]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-03T18:43:24.576 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:24 smithi113 ceph-mon[33540]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-03T18:43:24.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:24 smithi032 ceph-mon[29763]: from='osd.10 [v2:172.21.15.45:6824/1817064361,v1:172.21.15.45:6825/1817064361]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-03T18:43:24.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:24 smithi032 ceph-mon[29763]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-03T18:43:25.670 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:25 smithi113 ceph-mon[33540]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-03T18:43:25.673 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:25 smithi113 ceph-mon[33540]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-03T18:43:25.673 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:25 smithi113 ceph-mon[33540]: osdmap e62: 11 total, 10 up, 11 in 2024-05-03T18:43:25.673 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:25 smithi113 ceph-mon[33540]: from='osd.10 [v2:172.21.15.45:6824/1817064361,v1:172.21.15.45:6825/1817064361]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-05-03T18:43:25.673 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:25 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-03T18:43:25.673 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:25 smithi113 ceph-mon[33540]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-05-03T18:43:25.705 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:25 smithi045 ceph-mon[33299]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-03T18:43:25.705 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:25 smithi045 ceph-mon[33299]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-03T18:43:25.705 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:25 smithi045 ceph-mon[33299]: osdmap e62: 11 total, 10 up, 11 in 2024-05-03T18:43:25.705 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:25 smithi045 ceph-mon[33299]: from='osd.10 [v2:172.21.15.45:6824/1817064361,v1:172.21.15.45:6825/1817064361]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-05-03T18:43:25.705 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:25 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-03T18:43:25.706 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:25 smithi045 ceph-mon[33299]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-05-03T18:43:25.706 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 18:43:25 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T18:43:25.415+0000 7fc74a230700 -1 osd.10 0 waiting for initial osdmap 2024-05-03T18:43:25.706 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 18:43:25 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T18:43:25.430+0000 7fc746023700 -1 osd.10 63 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-03T18:43:25.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:25 smithi032 ceph-mon[29763]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-03T18:43:25.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:25 smithi032 ceph-mon[29763]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-03T18:43:25.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:25 smithi032 ceph-mon[29763]: osdmap e62: 11 total, 10 up, 11 in 2024-05-03T18:43:25.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:25 smithi032 ceph-mon[29763]: from='osd.10 [v2:172.21.15.45:6824/1817064361,v1:172.21.15.45:6825/1817064361]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-05-03T18:43:25.819 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:25 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-03T18:43:25.819 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:25 smithi032 ceph-mon[29763]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-05-03T18:43:26.239 INFO:teuthology.orchestra.run.smithi113.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-03T18:43:26.239 INFO:teuthology.orchestra.run.smithi113.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-03T18:43:26.239 INFO:teuthology.orchestra.run.smithi113.stderr: stderr: 10+0 records in 2024-05-03T18:43:26.239 INFO:teuthology.orchestra.run.smithi113.stderr:10+0 records out 2024-05-03T18:43:26.239 INFO:teuthology.orchestra.run.smithi113.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00952074 s, 1.1 GB/s 2024-05-03T18:43:26.239 INFO:teuthology.orchestra.run.smithi113.stderr:--> Zapping successful for: 2024-05-03T18:43:26.591 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:26 smithi045 ceph-mon[33299]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2024-05-03T18:43:26.592 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:26 smithi045 ceph-mon[33299]: osdmap e63: 11 total, 10 up, 11 in 2024-05-03T18:43:26.592 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:26 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-03T18:43:26.592 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:26 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-03T18:43:26.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:26 smithi113 ceph-mon[33540]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2024-05-03T18:43:26.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:26 smithi113 ceph-mon[33540]: osdmap e63: 11 total, 10 up, 11 in 2024-05-03T18:43:26.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:26 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-03T18:43:26.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:26 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-03T18:43:26.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:26 smithi032 ceph-mon[29763]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2024-05-03T18:43:26.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:26 smithi032 ceph-mon[29763]: osdmap e63: 11 total, 10 up, 11 in 2024-05-03T18:43:26.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:26 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-03T18:43:26.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:26 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-03T18:43:26.934 DEBUG:teuthology.orchestra.run.smithi113:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f500455e-097b-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi113:vg_nvme/lv_1 2024-05-03T18:43:27.180 INFO:teuthology.orchestra.run.smithi113.stderr:Inferring config /var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/mon.c/config 2024-05-03T18:43:27.533 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:27 smithi113 ceph-mon[33540]: purged_snaps scrub starts 2024-05-03T18:43:27.533 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:27 smithi113 ceph-mon[33540]: purged_snaps scrub ok 2024-05-03T18:43:27.533 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:27 smithi113 ceph-mon[33540]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-03T18:43:27.533 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:27 smithi113 ceph-mon[33540]: osd.10 [v2:172.21.15.45:6824/1817064361,v1:172.21.15.45:6825/1817064361] boot 2024-05-03T18:43:27.533 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:27 smithi113 ceph-mon[33540]: osdmap e64: 11 total, 11 up, 11 in 2024-05-03T18:43:27.533 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:27 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-03T18:43:27.533 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:27 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:43:27.534 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:27 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:43:27.534 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:27 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:43:27.534 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:27 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:43:27.534 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:27 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:43:27.534 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:27 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:43:27.534 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:27 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:43:27.534 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:27 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:43:27.534 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:27 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:43:27.534 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:27 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:43:27.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:27 smithi045 ceph-mon[33299]: purged_snaps scrub starts 2024-05-03T18:43:27.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:27 smithi045 ceph-mon[33299]: purged_snaps scrub ok 2024-05-03T18:43:27.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:27 smithi045 ceph-mon[33299]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-03T18:43:27.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:27 smithi045 ceph-mon[33299]: osd.10 [v2:172.21.15.45:6824/1817064361,v1:172.21.15.45:6825/1817064361] boot 2024-05-03T18:43:27.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:27 smithi045 ceph-mon[33299]: osdmap e64: 11 total, 11 up, 11 in 2024-05-03T18:43:27.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:27 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-03T18:43:27.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:27 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:43:27.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:27 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:43:27.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:27 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:43:27.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:27 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:43:27.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:27 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:43:27.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:27 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:43:27.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:27 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:43:27.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:27 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:43:27.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:27 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:43:27.749 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:27 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:43:27.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:27 smithi032 ceph-mon[29763]: purged_snaps scrub starts 2024-05-03T18:43:27.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:27 smithi032 ceph-mon[29763]: purged_snaps scrub ok 2024-05-03T18:43:27.819 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:27 smithi032 ceph-mon[29763]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-05-03T18:43:27.819 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:27 smithi032 ceph-mon[29763]: osd.10 [v2:172.21.15.45:6824/1817064361,v1:172.21.15.45:6825/1817064361] boot 2024-05-03T18:43:27.819 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:27 smithi032 ceph-mon[29763]: osdmap e64: 11 total, 11 up, 11 in 2024-05-03T18:43:27.819 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:27 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-03T18:43:27.819 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:27 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:43:27.819 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:27 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:43:27.819 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:27 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:43:27.819 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:27 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:43:27.819 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:27 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:43:27.819 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:27 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:43:27.819 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:27 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:43:27.819 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:27 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:43:27.819 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:27 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:43:27.819 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:27 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:43:28.531 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:28 smithi113 ceph-mon[33540]: Detected new or changed devices on smithi045 2024-05-03T18:43:28.531 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:28 smithi113 ceph-mon[33540]: Adjusting osd_memory_target on smithi045 to 4273M 2024-05-03T18:43:28.531 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:28 smithi113 ceph-mon[33540]: osdmap e65: 11 total, 11 up, 11 in 2024-05-03T18:43:28.531 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:28 smithi113 ceph-mon[33540]: osdmap e66: 11 total, 11 up, 11 in 2024-05-03T18:43:28.542 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:28 smithi045 ceph-mon[33299]: Detected new or changed devices on smithi045 2024-05-03T18:43:28.542 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:28 smithi045 ceph-mon[33299]: Adjusting osd_memory_target on smithi045 to 4273M 2024-05-03T18:43:28.542 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:28 smithi045 ceph-mon[33299]: osdmap e65: 11 total, 11 up, 11 in 2024-05-03T18:43:28.542 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:28 smithi045 ceph-mon[33299]: osdmap e66: 11 total, 11 up, 11 in 2024-05-03T18:43:28.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:43:28.782+0000 7ff693338700 1 -- 172.21.15.113:0/2827297879 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff68c101bb0 msgr2=0x7ff68c101fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:43:28.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:43:28.782+0000 7ff693338700 1 --2- 172.21.15.113:0/2827297879 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff68c101bb0 0x7ff68c101fd0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7ff688009990 tx=0x7ff688009ca0 comp rx=0 tx=0).stop 2024-05-03T18:43:28.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:43:28.782+0000 7ff693338700 1 -- 172.21.15.113:0/2827297879 shutdown_connections 2024-05-03T18:43:28.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:43:28.782+0000 7ff693338700 1 --2- 172.21.15.113:0/2827297879 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff68c103750 0x7ff68c10a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:28.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:43:28.782+0000 7ff693338700 1 --2- 172.21.15.113:0/2827297879 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff68c101bb0 0x7ff68c101fd0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:28.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:43:28.782+0000 7ff693338700 1 --2- 172.21.15.113:0/2827297879 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff68c102db0 0x7ff68c103210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:28.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:43:28.782+0000 7ff693338700 1 -- 172.21.15.113:0/2827297879 >> 172.21.15.113:0/2827297879 conn(0x7ff68c0fd150 msgr2=0x7ff68c0ff590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:43:28.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:43:28.782+0000 7ff693338700 1 -- 172.21.15.113:0/2827297879 shutdown_connections 2024-05-03T18:43:28.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:43:28.782+0000 7ff693338700 1 -- 172.21.15.113:0/2827297879 wait complete. 2024-05-03T18:43:28.785 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:43:28.782+0000 7ff693338700 1 Processor -- start 2024-05-03T18:43:28.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:43:28.782+0000 7ff693338700 1 -- start start 2024-05-03T18:43:28.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:43:28.783+0000 7ff693338700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff68c101bb0 0x7ff68c1985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:43:28.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:43:28.783+0000 7ff693338700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff68c102db0 0x7ff68c198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:43:28.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:43:28.783+0000 7ff693338700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff68c103750 0x7ff68c19fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:43:28.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:43:28.783+0000 7ff693338700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff68c10d640 con 0x7ff68c103750 2024-05-03T18:43:28.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:43:28.783+0000 7ff693338700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff68c10d4c0 con 0x7ff68c101bb0 2024-05-03T18:43:28.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:43:28.783+0000 7ff693338700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7ff68c10d7c0 con 0x7ff68c102db0 2024-05-03T18:43:28.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:43:28.783+0000 7ff6908d3700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff68c102db0 0x7ff68c198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:43:28.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:43:28.783+0000 7ff6910d4700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff68c101bb0 0x7ff68c1985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:43:28.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:43:28.783+0000 7ff6908d3700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff68c102db0 0x7ff68c198b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:36820/0 (socket says 172.21.15.113:36820) 2024-05-03T18:43:28.786 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:43:28.783+0000 7ff6908d3700 1 -- 172.21.15.113:0/1007275571 learned_addr learned my addr 172.21.15.113:0/1007275571 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-05-03T18:43:28.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:43:28.783+0000 7ff6918d5700 1 --2- 172.21.15.113:0/1007275571 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff68c103750 0x7ff68c19fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:43:28.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:43:28.783+0000 7ff6908d3700 1 -- 172.21.15.113:0/1007275571 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff68c101bb0 msgr2=0x7ff68c1985c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:43:28.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:43:28.783+0000 7ff6908d3700 1 --2- 172.21.15.113:0/1007275571 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff68c101bb0 0x7ff68c1985c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:28.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:43:28.783+0000 7ff6908d3700 1 -- 172.21.15.113:0/1007275571 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff68c103750 msgr2=0x7ff68c19fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:43:28.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:43:28.783+0000 7ff6908d3700 1 --2- 172.21.15.113:0/1007275571 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff68c103750 0x7ff68c19fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:28.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:43:28.783+0000 7ff6908d3700 1 -- 172.21.15.113:0/1007275571 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff68c0fe8c0 con 0x7ff68c102db0 2024-05-03T18:43:28.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:43:28.783+0000 7ff6910d4700 1 --2- 172.21.15.113:0/1007275571 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff68c101bb0 0x7ff68c1985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T18:43:28.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:43:28.783+0000 7ff6918d5700 1 --2- 172.21.15.113:0/1007275571 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff68c103750 0x7ff68c19fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T18:43:28.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:43:28.783+0000 7ff6908d3700 1 --2- 172.21.15.113:0/1007275571 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff68c102db0 0x7ff68c198b00 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7ff67c00e9c0 tx=0x7ff67c00ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:43:28.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:43:28.783+0000 7ff6827fc700 1 -- 172.21.15.113:0/1007275571 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff67c00cbd0 con 0x7ff68c102db0 2024-05-03T18:43:28.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:43:28.784+0000 7ff693338700 1 -- 172.21.15.113:0/1007275571 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff68c0febb0 con 0x7ff68c102db0 2024-05-03T18:43:28.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:43:28.784+0000 7ff6827fc700 1 -- 172.21.15.113:0/1007275571 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff67c00cd70 con 0x7ff68c102db0 2024-05-03T18:43:28.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:43:28.784+0000 7ff6827fc700 1 -- 172.21.15.113:0/1007275571 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff67c005610 con 0x7ff68c102db0 2024-05-03T18:43:28.787 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:43:28.784+0000 7ff693338700 1 -- 172.21.15.113:0/1007275571 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff68c0ff0f0 con 0x7ff68c102db0 2024-05-03T18:43:28.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:43:28.785+0000 7ff693338700 1 -- 172.21.15.113:0/1007275571 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff68c06a190 con 0x7ff68c102db0 2024-05-03T18:43:28.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:43:28.786+0000 7ff6827fc700 1 -- 172.21.15.113:0/1007275571 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7ff67c004750 con 0x7ff68c102db0 2024-05-03T18:43:28.788 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:43:28.786+0000 7ff6827fc700 1 --2- 172.21.15.113:0/1007275571 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7ff6780b2880 0x7ff6780b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:43:28.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:43:28.787+0000 7ff6827fc700 1 -- 172.21.15.113:0/1007275571 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6694+0+0 (secure 0 0 0) 0x7ff67c0e2120 con 0x7ff68c102db0 2024-05-03T18:43:28.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:43:28.787+0000 7ff6910d4700 1 --2- 172.21.15.113:0/1007275571 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7ff6780b2880 0x7ff6780b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:43:28.789 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:43:28.787+0000 7ff6910d4700 1 --2- 172.21.15.113:0/1007275571 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7ff6780b2880 0x7ff6780b4d40 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ff68800b720 tx=0x7ff68801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:43:28.791 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:43:28.790+0000 7ff6827fc700 1 -- 172.21.15.113:0/1007275571 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7ff67c054d70 con 0x7ff68c102db0 2024-05-03T18:43:28.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:28 smithi032 ceph-mon[29763]: Detected new or changed devices on smithi045 2024-05-03T18:43:28.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:28 smithi032 ceph-mon[29763]: Adjusting osd_memory_target on smithi045 to 4273M 2024-05-03T18:43:28.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:28 smithi032 ceph-mon[29763]: osdmap e65: 11 total, 11 up, 11 in 2024-05-03T18:43:28.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:28 smithi032 ceph-mon[29763]: osdmap e66: 11 total, 11 up, 11 in 2024-05-03T18:43:29.001 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:43:29.000+0000 7ff693338700 1 -- 172.21.15.113:0/1007275571 --> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi113:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7ff68c02e580 con 0x7ff6780b2880 2024-05-03T18:43:29.717 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:29 smithi113 ceph-mon[33540]: pgmap v226: 1 pgs: 1 peering; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-05-03T18:43:29.717 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:29 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T18:43:29.717 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:29 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T18:43:29.717 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:29 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:43:29.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:29 smithi045 ceph-mon[33299]: pgmap v226: 1 pgs: 1 peering; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-05-03T18:43:29.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:29 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T18:43:29.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:29 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T18:43:29.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:29 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:43:29.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:29 smithi032 ceph-mon[29763]: pgmap v226: 1 pgs: 1 peering; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-05-03T18:43:29.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:29 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-03T18:43:29.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:29 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-03T18:43:29.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:29 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:43:30.747 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:30 smithi045 ceph-mon[33299]: from='client.24379 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi113:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:43:30.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:30 smithi113 ceph-mon[33540]: from='client.24379 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi113:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:43:30.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:30 smithi032 ceph-mon[29763]: from='client.24379 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi113:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:43:31.747 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:31 smithi045 ceph-mon[33299]: pgmap v228: 1 pgs: 1 peering; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-05-03T18:43:31.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:31 smithi113 ceph-mon[33540]: pgmap v228: 1 pgs: 1 peering; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-05-03T18:43:31.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:31 smithi032 ceph-mon[29763]: pgmap v228: 1 pgs: 1 peering; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-05-03T18:43:32.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:32 smithi045 ceph-mon[33299]: from='client.? 172.21.15.113:0/2383382400' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "22981709-1f9d-46f6-892a-5678394a2e1f"}]: dispatch 2024-05-03T18:43:32.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:32 smithi045 ceph-mon[33299]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "22981709-1f9d-46f6-892a-5678394a2e1f"}]: dispatch 2024-05-03T18:43:32.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:32 smithi045 ceph-mon[33299]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "22981709-1f9d-46f6-892a-5678394a2e1f"}]': finished 2024-05-03T18:43:32.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:32 smithi045 ceph-mon[33299]: osdmap e67: 12 total, 11 up, 12 in 2024-05-03T18:43:32.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:32 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-03T18:43:32.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:32 smithi113 ceph-mon[33540]: from='client.? 172.21.15.113:0/2383382400' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "22981709-1f9d-46f6-892a-5678394a2e1f"}]: dispatch 2024-05-03T18:43:32.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:32 smithi113 ceph-mon[33540]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "22981709-1f9d-46f6-892a-5678394a2e1f"}]: dispatch 2024-05-03T18:43:32.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:32 smithi113 ceph-mon[33540]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "22981709-1f9d-46f6-892a-5678394a2e1f"}]': finished 2024-05-03T18:43:32.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:32 smithi113 ceph-mon[33540]: osdmap e67: 12 total, 11 up, 12 in 2024-05-03T18:43:32.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:32 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-03T18:43:32.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:32 smithi032 ceph-mon[29763]: from='client.? 172.21.15.113:0/2383382400' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "22981709-1f9d-46f6-892a-5678394a2e1f"}]: dispatch 2024-05-03T18:43:32.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:32 smithi032 ceph-mon[29763]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "22981709-1f9d-46f6-892a-5678394a2e1f"}]: dispatch 2024-05-03T18:43:32.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:32 smithi032 ceph-mon[29763]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "22981709-1f9d-46f6-892a-5678394a2e1f"}]': finished 2024-05-03T18:43:32.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:32 smithi032 ceph-mon[29763]: osdmap e67: 12 total, 11 up, 12 in 2024-05-03T18:43:32.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:32 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-03T18:43:33.747 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:33 smithi045 ceph-mon[33299]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 104 KiB/s, 0 objects/s recovering 2024-05-03T18:43:33.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:33 smithi045 ceph-mon[33299]: from='client.? 172.21.15.113:0/821273856' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T18:43:33.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:33 smithi113 ceph-mon[33540]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 104 KiB/s, 0 objects/s recovering 2024-05-03T18:43:33.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:33 smithi113 ceph-mon[33540]: from='client.? 172.21.15.113:0/821273856' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T18:43:33.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:33 smithi032 ceph-mon[29763]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 104 KiB/s, 0 objects/s recovering 2024-05-03T18:43:33.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:33 smithi032 ceph-mon[29763]: from='client.? 172.21.15.113:0/821273856' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-03T18:43:35.747 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:35 smithi045 ceph-mon[33299]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-05-03T18:43:35.751 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:35 smithi113 ceph-mon[33540]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-05-03T18:43:35.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:35 smithi032 ceph-mon[29763]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-05-03T18:43:37.747 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:37 smithi045 ceph-mon[33299]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-03T18:43:37.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:37 smithi113 ceph-mon[33540]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-03T18:43:37.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:37 smithi032 ceph-mon[29763]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-03T18:43:39.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:39 smithi045 ceph-mon[33299]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-05-03T18:43:39.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:39 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-03T18:43:39.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:39 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:43:39.800 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:39 smithi113 ceph-mon[33540]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-05-03T18:43:39.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:39 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-03T18:43:39.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:39 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:43:39.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:39 smithi032 ceph-mon[29763]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-05-03T18:43:39.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:39 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-03T18:43:39.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:39 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:43:40.708 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:40 smithi113 ceph-mon[33540]: Deploying daemon osd.11 on smithi113 2024-05-03T18:43:40.747 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:40 smithi045 ceph-mon[33299]: Deploying daemon osd.11 on smithi113 2024-05-03T18:43:40.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:40 smithi032 ceph-mon[29763]: Deploying daemon osd.11 on smithi113 2024-05-03T18:43:41.653 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:41 smithi113 ceph-mon[33540]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-03T18:43:41.747 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:41 smithi045 ceph-mon[33299]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-03T18:43:41.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:41 smithi032 ceph-mon[29763]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-03T18:43:43.708 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:43 smithi113 ceph-mon[33540]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-05-03T18:43:43.747 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:43 smithi045 ceph-mon[33299]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-05-03T18:43:43.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:43 smithi032 ceph-mon[29763]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-05-03T18:43:45.667 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:45 smithi113 ceph-mon[33540]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-05-03T18:43:45.667 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:45 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:43:45.667 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:45 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:43:45.667 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:45 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:43:45.747 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:45 smithi045 ceph-mon[33299]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-05-03T18:43:45.747 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:45 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:43:45.747 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:45 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:43:45.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:45 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:43:45.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:45 smithi032 ceph-mon[29763]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-05-03T18:43:45.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:45 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:43:45.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:45 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:43:45.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:45 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:43:47.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:47 smithi113 ceph-mon[33540]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-05-03T18:43:47.747 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:47 smithi045 ceph-mon[33299]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-05-03T18:43:47.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:47 smithi032 ceph-mon[29763]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-05-03T18:43:47.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:43:47.974+0000 7ff6827fc700 1 -- 172.21.15.113:0/1007275571 <== mgr.14152 v2:172.21.15.32:6800/1796547904 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7ff68c02e580 con 0x7ff6780b2880 2024-05-03T18:43:47.975 INFO:teuthology.orchestra.run.smithi113.stdout:Created osd(s) 11 on host 'smithi113' 2024-05-03T18:43:47.977 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:43:47.977+0000 7ff693338700 1 -- 172.21.15.113:0/1007275571 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7ff6780b2880 msgr2=0x7ff6780b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:43:47.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:43:47.977+0000 7ff693338700 1 --2- 172.21.15.113:0/1007275571 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7ff6780b2880 0x7ff6780b4d40 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ff68800b720 tx=0x7ff68801a040 comp rx=0 tx=0).stop 2024-05-03T18:43:47.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:43:47.977+0000 7ff693338700 1 -- 172.21.15.113:0/1007275571 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff68c102db0 msgr2=0x7ff68c198b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:43:47.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:43:47.977+0000 7ff693338700 1 --2- 172.21.15.113:0/1007275571 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff68c102db0 0x7ff68c198b00 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7ff67c00e9c0 tx=0x7ff67c00ecd0 comp rx=0 tx=0).stop 2024-05-03T18:43:47.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:43:47.977+0000 7ff693338700 1 -- 172.21.15.113:0/1007275571 shutdown_connections 2024-05-03T18:43:47.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:43:47.977+0000 7ff693338700 1 --2- 172.21.15.113:0/1007275571 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7ff6780b2880 0x7ff6780b4d40 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:47.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:43:47.977+0000 7ff693338700 1 --2- 172.21.15.113:0/1007275571 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff68c103750 0x7ff68c19fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:47.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:43:47.977+0000 7ff693338700 1 --2- 172.21.15.113:0/1007275571 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff68c101bb0 0x7ff68c1985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:47.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:43:47.977+0000 7ff693338700 1 --2- 172.21.15.113:0/1007275571 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff68c102db0 0x7ff68c198b00 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:47.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:43:47.977+0000 7ff693338700 1 -- 172.21.15.113:0/1007275571 >> 172.21.15.113:0/1007275571 conn(0x7ff68c0fd150 msgr2=0x7ff68c106870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:43:47.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:43:47.977+0000 7ff693338700 1 -- 172.21.15.113:0/1007275571 shutdown_connections 2024-05-03T18:43:47.978 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:43:47.977+0000 7ff693338700 1 -- 172.21.15.113:0/1007275571 wait complete. 2024-05-03T18:43:48.848 DEBUG:teuthology.orchestra.run.smithi113:osd.11> sudo journalctl -f -n 0 -u ceph-f500455e-097b-11ef-bc97-c7b262605968@osd.11.service 2024-05-03T18:43:48.851 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-05-03T18:43:48.851 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f500455e-097b-11ef-bc97-c7b262605968 -- ceph osd stat -f json 2024-05-03T18:43:49.112 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:43:49.235 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:48 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:43:49.235 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:48 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:43:49.235 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:48 smithi032 ceph-mon[29763]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-05-03T18:43:49.235 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:48 smithi032 ceph-mon[29763]: from='osd.11 [v2:172.21.15.113:6824/703464306,v1:172.21.15.113:6825/703464306]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-03T18:43:49.235 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:48 smithi032 ceph-mon[29763]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-03T18:43:49.247 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:48 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:43:49.247 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:48 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:43:49.248 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:48 smithi045 ceph-mon[33299]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-05-03T18:43:49.248 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:48 smithi045 ceph-mon[33299]: from='osd.11 [v2:172.21.15.113:6824/703464306,v1:172.21.15.113:6825/703464306]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-03T18:43:49.248 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:48 smithi045 ceph-mon[33299]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-03T18:43:49.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:48 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:43:49.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:48 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:43:49.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:48 smithi113 ceph-mon[33540]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-05-03T18:43:49.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:48 smithi113 ceph-mon[33540]: from='osd.11 [v2:172.21.15.113:6824/703464306,v1:172.21.15.113:6825/703464306]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-03T18:43:49.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:48 smithi113 ceph-mon[33540]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-03T18:43:50.010 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:49 smithi113 ceph-mon[33540]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-03T18:43:50.010 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:49 smithi113 ceph-mon[33540]: osdmap e68: 12 total, 11 up, 12 in 2024-05-03T18:43:50.010 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:49 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-03T18:43:50.010 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:49 smithi113 ceph-mon[33540]: from='osd.11 [v2:172.21.15.113:6824/703464306,v1:172.21.15.113:6825/703464306]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi113", "root=default"]}]: dispatch 2024-05-03T18:43:50.010 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:49 smithi113 ceph-mon[33540]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi113", "root=default"]}]: dispatch 2024-05-03T18:43:50.010 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:49 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:43:50.011 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:49 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:43:50.011 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 18:43:49 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T18:43:49.937+0000 7fe1bd43c700 -1 osd.11 0 waiting for initial osdmap 2024-05-03T18:43:50.011 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 18:43:49 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T18:43:49.950+0000 7fe1b8a2e700 -1 osd.11 69 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-03T18:43:50.247 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:49 smithi045 ceph-mon[33299]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-03T18:43:50.248 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:49 smithi045 ceph-mon[33299]: osdmap e68: 12 total, 11 up, 12 in 2024-05-03T18:43:50.248 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:49 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-03T18:43:50.248 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:49 smithi045 ceph-mon[33299]: from='osd.11 [v2:172.21.15.113:6824/703464306,v1:172.21.15.113:6825/703464306]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi113", "root=default"]}]: dispatch 2024-05-03T18:43:50.248 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:49 smithi045 ceph-mon[33299]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi113", "root=default"]}]: dispatch 2024-05-03T18:43:50.248 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:49 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:43:50.248 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:49 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:43:50.266 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:49 smithi032 ceph-mon[29763]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-03T18:43:50.266 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:49 smithi032 ceph-mon[29763]: osdmap e68: 12 total, 11 up, 12 in 2024-05-03T18:43:50.266 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:49 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-03T18:43:50.266 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:49 smithi032 ceph-mon[29763]: from='osd.11 [v2:172.21.15.113:6824/703464306,v1:172.21.15.113:6825/703464306]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi113", "root=default"]}]: dispatch 2024-05-03T18:43:50.267 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:49 smithi032 ceph-mon[29763]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi113", "root=default"]}]: dispatch 2024-05-03T18:43:50.267 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:49 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:43:50.267 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:49 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:43:50.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:50.828+0000 7f314c8e7700 1 -- 172.21.15.32:0/2056971229 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3148103600 msgr2=0x7f3148103a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:43:50.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:50.828+0000 7f314c8e7700 1 --2- 172.21.15.32:0/2056971229 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3148103600 0x7f3148103a60 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f313c0099c0 tx=0x7f313c009cd0 comp rx=0 tx=0).stop 2024-05-03T18:43:50.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:50.829+0000 7f314c8e7700 1 -- 172.21.15.32:0/2056971229 shutdown_connections 2024-05-03T18:43:50.830 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:50.829+0000 7f314c8e7700 1 --2- 172.21.15.32:0/2056971229 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3148103fa0 0x7f314810a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:50.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:50.829+0000 7f314c8e7700 1 --2- 172.21.15.32:0/2056971229 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3148102400 0x7f3148102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:50.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:50.829+0000 7f314c8e7700 1 --2- 172.21.15.32:0/2056971229 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3148103600 0x7f3148103a60 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:50.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:50.829+0000 7f314c8e7700 1 -- 172.21.15.32:0/2056971229 >> 172.21.15.32:0/2056971229 conn(0x7f31480fd9a0 msgr2=0x7f31480ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:43:50.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:50.829+0000 7f314c8e7700 1 -- 172.21.15.32:0/2056971229 shutdown_connections 2024-05-03T18:43:50.831 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:50.829+0000 7f314c8e7700 1 -- 172.21.15.32:0/2056971229 wait complete. 2024-05-03T18:43:50.833 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:50.829+0000 7f314c8e7700 1 Processor -- start 2024-05-03T18:43:50.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:50.830+0000 7f314c8e7700 1 -- start start 2024-05-03T18:43:50.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:50.830+0000 7f314c8e7700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3148102400 0x7f314806bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:43:50.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:50.830+0000 7f314c8e7700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3148103600 0x7f314819e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:43:50.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:50.830+0000 7f314c8e7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3148103fa0 0x7f31481a09b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:43:50.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:50.830+0000 7f314c8e7700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f314810de80 con 0x7f3148103fa0 2024-05-03T18:43:50.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:50.830+0000 7f314c8e7700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f314810dd00 con 0x7f3148103600 2024-05-03T18:43:50.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:50.830+0000 7f314c8e7700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f314810e000 con 0x7f3148102400 2024-05-03T18:43:50.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:50.830+0000 7f3146d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3148103fa0 0x7f31481a09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:43:50.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:50.830+0000 7f3146d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3148103fa0 0x7f31481a09b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38618/0 (socket says 172.21.15.32:38618) 2024-05-03T18:43:50.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:50.830+0000 7f3146d9d700 1 -- 172.21.15.32:0/1933548413 learned_addr learned my addr 172.21.15.32:0/1933548413 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:43:50.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:50.830+0000 7f314659c700 1 --2- 172.21.15.32:0/1933548413 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3148102400 0x7f314806bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:43:50.834 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:50.830+0000 7f3145d9b700 1 --2- 172.21.15.32:0/1933548413 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3148103600 0x7f314819e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:43:50.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:50.830+0000 7f3146d9d700 1 -- 172.21.15.32:0/1933548413 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3148102400 msgr2=0x7f314806bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:43:50.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:50.830+0000 7f3146d9d700 1 --2- 172.21.15.32:0/1933548413 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3148102400 0x7f314806bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:50.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:50.830+0000 7f3146d9d700 1 -- 172.21.15.32:0/1933548413 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3148103600 msgr2=0x7f314819e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:43:50.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:50.830+0000 7f3146d9d700 1 --2- 172.21.15.32:0/1933548413 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3148103600 0x7f314819e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:50.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:50.830+0000 7f3146d9d700 1 -- 172.21.15.32:0/1933548413 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31481a1280 con 0x7f3148103fa0 2024-05-03T18:43:50.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:50.830+0000 7f3145d9b700 1 --2- 172.21.15.32:0/1933548413 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3148103600 0x7f314819e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T18:43:50.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:50.830+0000 7f3146d9d700 1 --2- 172.21.15.32:0/1933548413 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3148103fa0 0x7f31481a09b0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f3140007d70 tx=0x7f314000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:43:50.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:50.831+0000 7f31377fe700 1 -- 172.21.15.32:0/1933548413 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3140016020 con 0x7f3148103fa0 2024-05-03T18:43:50.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:50.831+0000 7f31377fe700 1 -- 172.21.15.32:0/1933548413 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f314000ad40 con 0x7f3148103fa0 2024-05-03T18:43:50.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:50.831+0000 7f314c8e7700 1 -- 172.21.15.32:0/1933548413 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31481a1570 con 0x7f3148103fa0 2024-05-03T18:43:50.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:50.831+0000 7f31377fe700 1 -- 172.21.15.32:0/1933548413 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31400115e0 con 0x7f3148103fa0 2024-05-03T18:43:50.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:50.831+0000 7f314c8e7700 1 -- 172.21.15.32:0/1933548413 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31481a1a60 con 0x7f3148103fa0 2024-05-03T18:43:50.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:50.832+0000 7f314c8e7700 1 -- 172.21.15.32:0/1933548413 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f314806a190 con 0x7f3148103fa0 2024-05-03T18:43:50.835 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:50.833+0000 7f31377fe700 1 -- 172.21.15.32:0/1933548413 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f3140019020 con 0x7f3148103fa0 2024-05-03T18:43:50.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:50.834+0000 7f31377fe700 1 --2- 172.21.15.32:0/1933548413 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f31300b2880 0x7f31300b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:43:50.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:50.834+0000 7f314659c700 1 --2- 172.21.15.32:0/1933548413 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f31300b2880 0x7f31300b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:43:50.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:50.834+0000 7f31377fe700 1 -- 172.21.15.32:0/1933548413 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6847+0+0 (secure 0 0 0) 0x7f31400e21e0 con 0x7f3148103fa0 2024-05-03T18:43:50.836 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:50.834+0000 7f314659c700 1 --2- 172.21.15.32:0/1933548413 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f31300b2880 0x7f31300b4d40 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f3148103460 tx=0x7f313800a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:43:50.838 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:50.837+0000 7f31377fe700 1 -- 172.21.15.32:0/1933548413 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f3140020020 con 0x7f3148103fa0 2024-05-03T18:43:51.001 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:50 smithi113 ceph-mon[33540]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi113", "root=default"]}]': finished 2024-05-03T18:43:51.001 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:50 smithi113 ceph-mon[33540]: osdmap e69: 12 total, 11 up, 12 in 2024-05-03T18:43:51.001 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:50 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-03T18:43:51.001 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:50 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-03T18:43:51.001 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:50 smithi113 ceph-mon[33540]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-05-03T18:43:51.054 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:51.053+0000 7f314c8e7700 1 -- 172.21.15.32:0/1933548413 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f314802e590 con 0x7f3148103fa0 2024-05-03T18:43:51.055 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:51.054+0000 7f31377fe700 1 -- 172.21.15.32:0/1933548413 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v70) ==== 74+0+133 (secure 0 0 0) 0x7f31400aa420 con 0x7f3148103fa0 2024-05-03T18:43:51.055 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-03T18:43:51.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:51.056+0000 7f314c8e7700 1 -- 172.21.15.32:0/1933548413 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f31300b2880 msgr2=0x7f31300b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:43:51.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:51.056+0000 7f314c8e7700 1 --2- 172.21.15.32:0/1933548413 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f31300b2880 0x7f31300b4d40 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f3148103460 tx=0x7f313800a380 comp rx=0 tx=0).stop 2024-05-03T18:43:51.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:51.056+0000 7f314c8e7700 1 -- 172.21.15.32:0/1933548413 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3148103fa0 msgr2=0x7f31481a09b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:43:51.057 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:51.056+0000 7f314c8e7700 1 --2- 172.21.15.32:0/1933548413 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3148103fa0 0x7f31481a09b0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f3140007d70 tx=0x7f314000a340 comp rx=0 tx=0).stop 2024-05-03T18:43:51.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:51.056+0000 7f314c8e7700 1 -- 172.21.15.32:0/1933548413 shutdown_connections 2024-05-03T18:43:51.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:51.056+0000 7f314c8e7700 1 --2- 172.21.15.32:0/1933548413 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f31300b2880 0x7f31300b4d40 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:51.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:51.056+0000 7f314c8e7700 1 --2- 172.21.15.32:0/1933548413 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3148103fa0 0x7f31481a09b0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:51.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:51.056+0000 7f314c8e7700 1 --2- 172.21.15.32:0/1933548413 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3148102400 0x7f314806bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:51.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:51.056+0000 7f314c8e7700 1 --2- 172.21.15.32:0/1933548413 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3148103600 0x7f314819e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:51.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:51.056+0000 7f314c8e7700 1 -- 172.21.15.32:0/1933548413 >> 172.21.15.32:0/1933548413 conn(0x7f31480fd9a0 msgr2=0x7f31481069d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:43:51.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:51.056+0000 7f314c8e7700 1 -- 172.21.15.32:0/1933548413 shutdown_connections 2024-05-03T18:43:51.058 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:51.056+0000 7f314c8e7700 1 -- 172.21.15.32:0/1933548413 wait complete. 2024-05-03T18:43:51.095 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:50 smithi032 ceph-mon[29763]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi113", "root=default"]}]': finished 2024-05-03T18:43:51.095 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:50 smithi032 ceph-mon[29763]: osdmap e69: 12 total, 11 up, 12 in 2024-05-03T18:43:51.095 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:50 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-03T18:43:51.095 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:50 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-03T18:43:51.095 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:50 smithi032 ceph-mon[29763]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-05-03T18:43:51.247 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:50 smithi045 ceph-mon[33299]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi113", "root=default"]}]': finished 2024-05-03T18:43:51.248 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:50 smithi045 ceph-mon[33299]: osdmap e69: 12 total, 11 up, 12 in 2024-05-03T18:43:51.248 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:50 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-03T18:43:51.248 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:50 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-03T18:43:51.248 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:50 smithi045 ceph-mon[33299]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-05-03T18:43:51.704 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":70,"num_osds":12,"num_up_osds":12,"osd_up_since":1714761830,"num_in_osds":12,"osd_in_since":1714761811,"num_remapped_pgs":0} 2024-05-03T18:43:51.705 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid f500455e-097b-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-03T18:43:51.955 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:43:52.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:51 smithi113 ceph-mon[33540]: purged_snaps scrub starts 2024-05-03T18:43:52.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:51 smithi113 ceph-mon[33540]: purged_snaps scrub ok 2024-05-03T18:43:52.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:51 smithi113 ceph-mon[33540]: osd.11 [v2:172.21.15.113:6824/703464306,v1:172.21.15.113:6825/703464306] boot 2024-05-03T18:43:52.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:51 smithi113 ceph-mon[33540]: osdmap e70: 12 total, 12 up, 12 in 2024-05-03T18:43:52.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:51 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-03T18:43:52.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:51 smithi113 ceph-mon[33540]: from='client.? 172.21.15.32:0/1933548413' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-03T18:43:52.210 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:51 smithi032 ceph-mon[29763]: purged_snaps scrub starts 2024-05-03T18:43:52.210 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:51 smithi032 ceph-mon[29763]: purged_snaps scrub ok 2024-05-03T18:43:52.210 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:51 smithi032 ceph-mon[29763]: osd.11 [v2:172.21.15.113:6824/703464306,v1:172.21.15.113:6825/703464306] boot 2024-05-03T18:43:52.210 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:51 smithi032 ceph-mon[29763]: osdmap e70: 12 total, 12 up, 12 in 2024-05-03T18:43:52.210 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:51 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-03T18:43:52.210 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:51 smithi032 ceph-mon[29763]: from='client.? 172.21.15.32:0/1933548413' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-03T18:43:52.251 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:51 smithi045 ceph-mon[33299]: purged_snaps scrub starts 2024-05-03T18:43:52.251 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:51 smithi045 ceph-mon[33299]: purged_snaps scrub ok 2024-05-03T18:43:52.251 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:51 smithi045 ceph-mon[33299]: osd.11 [v2:172.21.15.113:6824/703464306,v1:172.21.15.113:6825/703464306] boot 2024-05-03T18:43:52.251 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:51 smithi045 ceph-mon[33299]: osdmap e70: 12 total, 12 up, 12 in 2024-05-03T18:43:52.251 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:51 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-03T18:43:52.251 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:51 smithi045 ceph-mon[33299]: from='client.? 172.21.15.32:0/1933548413' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-03T18:43:53.058 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:52 smithi032 ceph-mon[29763]: osdmap e71: 12 total, 12 up, 12 in 2024-05-03T18:43:53.059 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:52 smithi032 ceph-mon[29763]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:43:53.059 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:52 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:43:53.059 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:52 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:43:53.059 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:52 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:43:53.059 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:52 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:43:53.059 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:52 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:43:53.059 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:52 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:43:53.059 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:52 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:43:53.059 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:52 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:43:53.059 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:52 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:43:53.059 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:52 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:43:53.059 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:52 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:43:53.059 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:52 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:43:53.059 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:52 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:43:53.059 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:52 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:43:53.248 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:52 smithi045 ceph-mon[33299]: osdmap e71: 12 total, 12 up, 12 in 2024-05-03T18:43:53.248 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:52 smithi045 ceph-mon[33299]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:43:53.248 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:52 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:43:53.248 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:52 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:43:53.248 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:52 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:43:53.248 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:52 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:43:53.248 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:52 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:43:53.248 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:52 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:43:53.248 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:52 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:43:53.248 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:52 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:43:53.248 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:52 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:43:53.249 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:52 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:43:53.249 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:52 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:43:53.249 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:52 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:43:53.249 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:52 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:43:53.249 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:52 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:43:53.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:52 smithi113 ceph-mon[33540]: osdmap e71: 12 total, 12 up, 12 in 2024-05-03T18:43:53.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:52 smithi113 ceph-mon[33540]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:43:53.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:52 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:43:53.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:52 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:43:53.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:52 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:43:53.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:52 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:43:53.302 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:52 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:43:53.302 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:52 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:43:53.302 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:52 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:43:53.302 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:52 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:43:53.302 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:52 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:43:53.302 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:52 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:43:53.302 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:52 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:43:53.302 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:52 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:43:53.302 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:52 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:43:53.302 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:52 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:43:53.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:53.531+0000 7f5d7ccf6700 1 -- 172.21.15.32:0/476090628 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d7806c630 msgr2=0x7f5d78110360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:43:53.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:53.531+0000 7f5d7ccf6700 1 --2- 172.21.15.32:0/476090628 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d7806c630 0x7f5d78110360 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f5d7000b0d0 tx=0x7f5d7000b3e0 comp rx=0 tx=0).stop 2024-05-03T18:43:53.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:53.532+0000 7f5d7ccf6700 1 -- 172.21.15.32:0/476090628 shutdown_connections 2024-05-03T18:43:53.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:53.532+0000 7f5d7ccf6700 1 --2- 172.21.15.32:0/476090628 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d7806c630 0x7f5d78110360 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:53.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:53.532+0000 7f5d7ccf6700 1 --2- 172.21.15.32:0/476090628 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5d78109590 0x7f5d78109970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:53.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:53.532+0000 7f5d7ccf6700 1 --2- 172.21.15.32:0/476090628 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5d7806bc70 0x7f5d7806c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:53.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:53.532+0000 7f5d7ccf6700 1 -- 172.21.15.32:0/476090628 >> 172.21.15.32:0/476090628 conn(0x7f5d78079440 msgr2=0x7f5d78079850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:43:53.533 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:53.532+0000 7f5d7ccf6700 1 -- 172.21.15.32:0/476090628 shutdown_connections 2024-05-03T18:43:53.534 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:53.532+0000 7f5d7ccf6700 1 -- 172.21.15.32:0/476090628 wait complete. 2024-05-03T18:43:53.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:53.533+0000 7f5d7ccf6700 1 Processor -- start 2024-05-03T18:43:53.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:53.533+0000 7f5d7ccf6700 1 -- start start 2024-05-03T18:43:53.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:53.533+0000 7f5d7ccf6700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5d7806bc70 0x7f5d7819f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:43:53.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:53.533+0000 7f5d7ccf6700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5d7806c630 0x7f5d7819fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:43:53.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:53.533+0000 7f5d7ccf6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d78109590 0x7f5d78199a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:43:53.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:53.533+0000 7f5d7ccf6700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f5d78113770 con 0x7f5d78109590 2024-05-03T18:43:53.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:53.533+0000 7f5d7ccf6700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f5d781135f0 con 0x7f5d7806bc70 2024-05-03T18:43:53.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:53.533+0000 7f5d7ccf6700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f5d781138f0 con 0x7f5d7806c630 2024-05-03T18:43:53.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:53.533+0000 7f5d76d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d78109590 0x7f5d78199a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:43:53.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:53.533+0000 7f5d76d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d78109590 0x7f5d78199a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38632/0 (socket says 172.21.15.32:38632) 2024-05-03T18:43:53.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:53.533+0000 7f5d76d9d700 1 -- 172.21.15.32:0/1162017408 learned_addr learned my addr 172.21.15.32:0/1162017408 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:43:53.536 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:53.533+0000 7f5d7659c700 1 --2- 172.21.15.32:0/1162017408 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5d7806bc70 0x7f5d7819f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:43:53.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:53.533+0000 7f5d75d9b700 1 --2- 172.21.15.32:0/1162017408 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5d7806c630 0x7f5d7819fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:43:53.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:53.533+0000 7f5d76d9d700 1 -- 172.21.15.32:0/1162017408 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5d7806c630 msgr2=0x7f5d7819fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:43:53.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:53.533+0000 7f5d76d9d700 1 --2- 172.21.15.32:0/1162017408 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5d7806c630 0x7f5d7819fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:53.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:53.533+0000 7f5d76d9d700 1 -- 172.21.15.32:0/1162017408 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5d7806bc70 msgr2=0x7f5d7819f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:43:53.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:53.533+0000 7f5d76d9d700 1 --2- 172.21.15.32:0/1162017408 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5d7806bc70 0x7f5d7819f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:53.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:53.533+0000 7f5d76d9d700 1 -- 172.21.15.32:0/1162017408 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d7819a1d0 con 0x7f5d78109590 2024-05-03T18:43:53.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:53.533+0000 7f5d7659c700 1 --2- 172.21.15.32:0/1162017408 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5d7806bc70 0x7f5d7819f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T18:43:53.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:53.534+0000 7f5d76d9d700 1 --2- 172.21.15.32:0/1162017408 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d78109590 0x7f5d78199a70 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f5d70009fd0 tx=0x7f5d70007ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:43:53.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:53.534+0000 7f5d677fe700 1 -- 172.21.15.32:0/1162017408 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d7001f070 con 0x7f5d78109590 2024-05-03T18:43:53.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:53.534+0000 7f5d7ccf6700 1 -- 172.21.15.32:0/1162017408 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d7819a460 con 0x7f5d78109590 2024-05-03T18:43:53.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:53.534+0000 7f5d677fe700 1 -- 172.21.15.32:0/1162017408 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d70003c00 con 0x7f5d78109590 2024-05-03T18:43:53.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:53.534+0000 7f5d677fe700 1 -- 172.21.15.32:0/1162017408 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d7001b8e0 con 0x7f5d78109590 2024-05-03T18:43:53.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:53.534+0000 7f5d7ccf6700 1 -- 172.21.15.32:0/1162017408 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d781a6780 con 0x7f5d78109590 2024-05-03T18:43:53.537 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:53.535+0000 7f5d7ccf6700 1 -- 172.21.15.32:0/1162017408 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d7810ba20 con 0x7f5d78109590 2024-05-03T18:43:53.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:53.536+0000 7f5d677fe700 1 -- 172.21.15.32:0/1162017408 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f5d70022070 con 0x7f5d78109590 2024-05-03T18:43:53.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:53.537+0000 7f5d677fe700 1 --2- 172.21.15.32:0/1162017408 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f5d600b2880 0x7f5d600b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:43:53.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:53.537+0000 7f5d7659c700 1 --2- 172.21.15.32:0/1162017408 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f5d600b2880 0x7f5d600b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:43:53.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:53.537+0000 7f5d677fe700 1 -- 172.21.15.32:0/1162017408 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f5d700e3010 con 0x7f5d78109590 2024-05-03T18:43:53.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:53.538+0000 7f5d7659c700 1 --2- 172.21.15.32:0/1162017408 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f5d600b2880 0x7f5d600b4d40 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f5d7819abb0 tx=0x7f5d68008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:43:53.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:53.541+0000 7f5d677fe700 1 -- 172.21.15.32:0/1162017408 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f5d700ab260 con 0x7f5d78109590 2024-05-03T18:43:53.761 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:53.760+0000 7f5d7ccf6700 1 -- 172.21.15.32:0/1162017408 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5d7819afa0 con 0x7f5d78109590 2024-05-03T18:43:53.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:53.761+0000 7f5d677fe700 1 -- 172.21.15.32:0/1162017408 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) ==== 74+0+18385 (secure 0 0 0) 0x7f5d70009cb0 con 0x7f5d78109590 2024-05-03T18:43:53.763 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-03T18:43:53.763 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":71,"fsid":"f500455e-097b-11ef-bc97-c7b262605968","created":"2024-05-03T18:36:43.750772+0000","modified":"2024-05-03T18:43:51.939691+0000","last_up_change":"2024-05-03T18:43:50.933818+0000","last_in_change":"2024-05-03T18:43:31.804386+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-03T18:40:07.981853+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0430c3c0-2c50-471c-9c74-2869a0627c74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":262324956},{"type":"v1","addr":"172.21.15.32:6803","nonce":262324956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":262324956},{"type":"v1","addr":"172.21.15.32:6805","nonce":262324956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":262324956},{"type":"v1","addr":"172.21.15.32:6809","nonce":262324956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":262324956},{"type":"v1","addr":"172.21.15.32:6807","nonce":262324956}]},"public_addr":"172.21.15.32:6803/262324956","cluster_addr":"172.21.15.32:6805/262324956","heartbeat_back_addr":"172.21.15.32:6809/262324956","heartbeat_front_addr":"172.21.15.32:6807/262324956","state":["exists","up"]},{"osd":1,"uuid":"ba8f0b67-1d7d-4d4c-8276-eff664d21f02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1209907302},{"type":"v1","addr":"172.21.15.45:6801","nonce":1209907302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1209907302},{"type":"v1","addr":"172.21.15.45:6803","nonce":1209907302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1209907302},{"type":"v1","addr":"172.21.15.45:6807","nonce":1209907302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1209907302},{"type":"v1","addr":"172.21.15.45:6805","nonce":1209907302}]},"public_addr":"172.21.15.45:6801/1209907302","cluster_addr":"172.21.15.45:6803/1209907302","heartbeat_back_addr":"172.21.15.45:6807/1209907302","heartbeat_front_addr":"172.21.15.45:6805/1209907302","state":["exists","up"]},{"osd":2,"uuid":"5624eabc-be47-4bb8-9d8e-b84734769c83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":2308128926},{"type":"v1","addr":"172.21.15.113:6801","nonce":2308128926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":2308128926},{"type":"v1","addr":"172.21.15.113:6803","nonce":2308128926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":2308128926},{"type":"v1","addr":"172.21.15.113:6807","nonce":2308128926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":2308128926},{"type":"v1","addr":"172.21.15.113:6805","nonce":2308128926}]},"public_addr":"172.21.15.113:6801/2308128926","cluster_addr":"172.21.15.113:6803/2308128926","heartbeat_back_addr":"172.21.15.113:6807/2308128926","heartbeat_front_addr":"172.21.15.113:6805/2308128926","state":["exists","up"]},{"osd":3,"uuid":"62c9c977-afe1-4112-beba-b25af7caa2f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":2606514063},{"type":"v1","addr":"172.21.15.32:6811","nonce":2606514063}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":2606514063},{"type":"v1","addr":"172.21.15.32:6813","nonce":2606514063}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":2606514063},{"type":"v1","addr":"172.21.15.32:6817","nonce":2606514063}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":2606514063},{"type":"v1","addr":"172.21.15.32:6815","nonce":2606514063}]},"public_addr":"172.21.15.32:6811/2606514063","cluster_addr":"172.21.15.32:6813/2606514063","heartbeat_back_addr":"172.21.15.32:6817/2606514063","heartbeat_front_addr":"172.21.15.32:6815/2606514063","state":["exists","up"]},{"osd":4,"uuid":"99ad449f-b0bc-4aa1-b0cf-4a7a458126e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":71932118},{"type":"v1","addr":"172.21.15.45:6809","nonce":71932118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":71932118},{"type":"v1","addr":"172.21.15.45:6811","nonce":71932118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":71932118},{"type":"v1","addr":"172.21.15.45:6815","nonce":71932118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":71932118},{"type":"v1","addr":"172.21.15.45:6813","nonce":71932118}]},"public_addr":"172.21.15.45:6809/71932118","cluster_addr":"172.21.15.45:6811/71932118","heartbeat_back_addr":"172.21.15.45:6815/71932118","heartbeat_front_addr":"172.21.15.45:6813/71932118","state":["exists","up"]},{"osd":5,"uuid":"a6688ade-47d7-4a52-807f-3c880a8ffcbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":4085018597},{"type":"v1","addr":"172.21.15.113:6809","nonce":4085018597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":4085018597},{"type":"v1","addr":"172.21.15.113:6811","nonce":4085018597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":4085018597},{"type":"v1","addr":"172.21.15.113:6815","nonce":4085018597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":4085018597},{"type":"v1","addr":"172.21.15.113:6813","nonce":4085018597}]},"public_addr":"172.21.15.113:6809/4085018597","cluster_addr":"172.21.15.113:6811/4085018597","heartbeat_back_addr":"172.21.15.113:6815/4085018597","heartbeat_front_addr":"172.21.15.113:6813/4085018597","state":["exists","up"]},{"osd":6,"uuid":"1d9ebda9-55c7-473a-9dc0-fc60845bbfe9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":791407796},{"type":"v1","addr":"172.21.15.32:6819","nonce":791407796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":791407796},{"type":"v1","addr":"172.21.15.32:6821","nonce":791407796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":791407796},{"type":"v1","addr":"172.21.15.32:6825","nonce":791407796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":791407796},{"type":"v1","addr":"172.21.15.32:6823","nonce":791407796}]},"public_addr":"172.21.15.32:6819/791407796","cluster_addr":"172.21.15.32:6821/791407796","heartbeat_back_addr":"172.21.15.32:6825/791407796","heartbeat_front_addr":"172.21.15.32:6823/791407796","state":["exists","up"]},{"osd":7,"uuid":"988cce65-23ed-4814-ac09-177eb2f89fe1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1032572438},{"type":"v1","addr":"172.21.15.45:6817","nonce":1032572438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1032572438},{"type":"v1","addr":"172.21.15.45:6819","nonce":1032572438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1032572438},{"type":"v1","addr":"172.21.15.45:6823","nonce":1032572438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1032572438},{"type":"v1","addr":"172.21.15.45:6821","nonce":1032572438}]},"public_addr":"172.21.15.45:6817/1032572438","cluster_addr":"172.21.15.45:6819/1032572438","heartbeat_back_addr":"172.21.15.45:6823/1032572438","heartbeat_front_addr":"172.21.15.45:6821/1032572438","state":["exists","up"]},{"osd":8,"uuid":"816e8ce8-5afa-4b52-89be-f7793ed12a0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3569181558},{"type":"v1","addr":"172.21.15.113:6817","nonce":3569181558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3569181558},{"type":"v1","addr":"172.21.15.113:6819","nonce":3569181558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3569181558},{"type":"v1","addr":"172.21.15.113:6823","nonce":3569181558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3569181558},{"type":"v1","addr":"172.21.15.113:6821","nonce":3569181558}]},"public_addr":"172.21.15.113:6817/3569181558","cluster_addr":"172.21.15.113:6819/3569181558","heartbeat_back_addr":"172.21.15.113:6823/3569181558","heartbeat_front_addr":"172.21.15.113:6821/3569181558","state":["exists","up"]},{"osd":9,"uuid":"1d35c435-36b8-40e8-a2d4-0efe904f1107","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":4290196345},{"type":"v1","addr":"172.21.15.32:6827","nonce":4290196345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":4290196345},{"type":"v1","addr":"172.21.15.32:6829","nonce":4290196345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":4290196345},{"type":"v1","addr":"172.21.15.32:6833","nonce":4290196345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":4290196345},{"type":"v1","addr":"172.21.15.32:6831","nonce":4290196345}]},"public_addr":"172.21.15.32:6827/4290196345","cluster_addr":"172.21.15.32:6829/4290196345","heartbeat_back_addr":"172.21.15.32:6833/4290196345","heartbeat_front_addr":"172.21.15.32:6831/4290196345","state":["exists","up"]},{"osd":10,"uuid":"9af7879e-aa8d-4ef8-b289-4b145b614820","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1817064361},{"type":"v1","addr":"172.21.15.45:6825","nonce":1817064361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1817064361},{"type":"v1","addr":"172.21.15.45:6827","nonce":1817064361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1817064361},{"type":"v1","addr":"172.21.15.45:6831","nonce":1817064361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1817064361},{"type":"v1","addr":"172.21.15.45:6829","nonce":1817064361}]},"public_addr":"172.21.15.45:6825/1817064361","cluster_addr":"172.21.15.45:6827/1817064361","heartbeat_back_addr":"172.21.15.45:6831/1817064361","heartbeat_front_addr":"172.21.15.45:6829/1817064361","state":["exists","up"]},{"osd":11,"uuid":"22981709-1f9d-46f6-892a-5678394a2e1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":703464306},{"type":"v1","addr":"172.21.15.113:6825","nonce":703464306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":703464306},{"type":"v1","addr":"172.21.15.113:6827","nonce":703464306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":703464306},{"type":"v1","addr":"172.21.15.113:6831","nonce":703464306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":703464306},{"type":"v1","addr":"172.21.15.113:6829","nonce":703464306}]},"public_addr":"172.21.15.113:6825/703464306","cluster_addr":"172.21.15.113:6827/703464306","heartbeat_back_addr":"172.21.15.113:6831/703464306","heartbeat_front_addr":"172.21.15.113:6829/703464306","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:39:18.455575+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:39:42.234466+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:40:05.843072+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:40:30.822533+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:40:54.952487+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:41:18.750184+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:41:43.403326+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:42:08.285730+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:42:32.861740+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:42:59.307277+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:43:24.509691+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:43:49.850077+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:0/743758274":"2024-05-04T18:37:45.879602+0000","172.21.15.32:6801/2019702098":"2024-05-04T18:37:45.879602+0000","172.21.15.32:6800/2019702098":"2024-05-04T18:37:45.879602+0000","172.21.15.32:0/3192385439":"2024-05-04T18:37:18.492319+0000","172.21.15.32:0/1450308494":"2024-05-04T18:37:18.492319+0000","172.21.15.32:0/3957699240":"2024-05-04T18:37:45.879602+0000","172.21.15.32:0/3310503393":"2024-05-04T18:37:18.492319+0000","172.21.15.32:6801/3945405684":"2024-05-04T18:37:18.492319+0000","172.21.15.32:6800/3945405684":"2024-05-04T18:37:18.492319+0000","172.21.15.32:0/2471164468":"2024-05-04T18:37:45.879602+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-03T18:43:53.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:53.764+0000 7f5d7ccf6700 1 -- 172.21.15.32:0/1162017408 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f5d600b2880 msgr2=0x7f5d600b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:43:53.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:53.764+0000 7f5d7ccf6700 1 --2- 172.21.15.32:0/1162017408 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f5d600b2880 0x7f5d600b4d40 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f5d7819abb0 tx=0x7f5d68008040 comp rx=0 tx=0).stop 2024-05-03T18:43:53.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:53.764+0000 7f5d7ccf6700 1 -- 172.21.15.32:0/1162017408 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d78109590 msgr2=0x7f5d78199a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:43:53.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:53.764+0000 7f5d7ccf6700 1 --2- 172.21.15.32:0/1162017408 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d78109590 0x7f5d78199a70 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f5d70009fd0 tx=0x7f5d70007ad0 comp rx=0 tx=0).stop 2024-05-03T18:43:53.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:53.764+0000 7f5d7ccf6700 1 -- 172.21.15.32:0/1162017408 shutdown_connections 2024-05-03T18:43:53.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:53.764+0000 7f5d7ccf6700 1 --2- 172.21.15.32:0/1162017408 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f5d600b2880 0x7f5d600b4d40 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:53.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:53.764+0000 7f5d7ccf6700 1 --2- 172.21.15.32:0/1162017408 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d78109590 0x7f5d78199a70 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:53.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:53.764+0000 7f5d7ccf6700 1 --2- 172.21.15.32:0/1162017408 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5d7806bc70 0x7f5d7819f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:53.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:53.764+0000 7f5d7ccf6700 1 --2- 172.21.15.32:0/1162017408 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5d7806c630 0x7f5d7819fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:53.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:53.764+0000 7f5d7ccf6700 1 -- 172.21.15.32:0/1162017408 >> 172.21.15.32:0/1162017408 conn(0x7f5d78079440 msgr2=0x7f5d78101010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:43:53.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:53.764+0000 7f5d7ccf6700 1 -- 172.21.15.32:0/1162017408 shutdown_connections 2024-05-03T18:43:53.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:53.764+0000 7f5d7ccf6700 1 -- 172.21.15.32:0/1162017408 wait complete. 2024-05-03T18:43:54.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:53 smithi032 ceph-mon[29763]: Detected new or changed devices on smithi113 2024-05-03T18:43:54.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:53 smithi032 ceph-mon[29763]: Adjusting osd_memory_target on smithi113 to 4272M 2024-05-03T18:43:54.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:53 smithi032 ceph-mon[29763]: from='client.? 172.21.15.32:0/1162017408' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T18:43:54.247 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:53 smithi045 ceph-mon[33299]: Detected new or changed devices on smithi113 2024-05-03T18:43:54.248 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:53 smithi045 ceph-mon[33299]: Adjusting osd_memory_target on smithi113 to 4272M 2024-05-03T18:43:54.248 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:53 smithi045 ceph-mon[33299]: from='client.? 172.21.15.32:0/1162017408' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T18:43:54.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:53 smithi113 ceph-mon[33540]: Detected new or changed devices on smithi113 2024-05-03T18:43:54.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:53 smithi113 ceph-mon[33540]: Adjusting osd_memory_target on smithi113 to 4272M 2024-05-03T18:43:54.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:53 smithi113 ceph-mon[33540]: from='client.? 172.21.15.32:0/1162017408' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T18:43:54.472 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-03T18:40:07.981853+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-03T18:43:54.473 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid f500455e-097b-11ef-bc97-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-05-03T18:43:54.721 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:43:55.110 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:55 smithi032 ceph-mon[29763]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:43:55.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:55 smithi113 ceph-mon[33540]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:43:55.497 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:55 smithi045 ceph-mon[33299]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:43:56.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:56.427+0000 7f991aa4c700 1 -- 172.21.15.32:0/2636954900 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99141023a0 msgr2=0x7f9914102820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:43:56.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:56.427+0000 7f991aa4c700 1 --2- 172.21.15.32:0/2636954900 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99141023a0 0x7f9914102820 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f9908009990 tx=0x7f9908009ca0 comp rx=0 tx=0).stop 2024-05-03T18:43:56.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:56.427+0000 7f991aa4c700 1 -- 172.21.15.32:0/2636954900 shutdown_connections 2024-05-03T18:43:56.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:56.427+0000 7f991aa4c700 1 --2- 172.21.15.32:0/2636954900 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9914102d60 0x7f991410f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:56.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:56.427+0000 7f991aa4c700 1 --2- 172.21.15.32:0/2636954900 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9914108410 0x7f99141087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:56.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:56.427+0000 7f991aa4c700 1 --2- 172.21.15.32:0/2636954900 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99141023a0 0x7f9914102820 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:56.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:56.427+0000 7f991aa4c700 1 -- 172.21.15.32:0/2636954900 >> 172.21.15.32:0/2636954900 conn(0x7f9914079350 msgr2=0x7f9914079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:43:56.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:56.427+0000 7f991aa4c700 1 -- 172.21.15.32:0/2636954900 shutdown_connections 2024-05-03T18:43:56.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:56.427+0000 7f991aa4c700 1 -- 172.21.15.32:0/2636954900 wait complete. 2024-05-03T18:43:56.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:56.428+0000 7f991aa4c700 1 Processor -- start 2024-05-03T18:43:56.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:56.428+0000 7f991aa4c700 1 -- start start 2024-05-03T18:43:56.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:56.428+0000 7f991aa4c700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f99141023a0 0x7f991419f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:43:56.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:56.428+0000 7f991aa4c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9914102d60 0x7f991419fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:43:56.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:56.428+0000 7f991aa4c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9914108410 0x7f9914199940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:43:56.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:56.428+0000 7f991aa4c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f9914112560 con 0x7f9914102d60 2024-05-03T18:43:56.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:56.428+0000 7f991aa4c700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f99141123e0 con 0x7f9914108410 2024-05-03T18:43:56.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:56.428+0000 7f991aa4c700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f99141126e0 con 0x7f99141023a0 2024-05-03T18:43:56.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:56.429+0000 7f99137fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9914102d60 0x7f991419fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:43:56.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:56.429+0000 7f9913fff700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f99141023a0 0x7f991419f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:43:56.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:56.429+0000 7f99137fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9914102d60 0x7f991419fcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38682/0 (socket says 172.21.15.32:38682) 2024-05-03T18:43:56.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:56.429+0000 7f99137fe700 1 -- 172.21.15.32:0/1157567493 learned_addr learned my addr 172.21.15.32:0/1157567493 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:43:56.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:56.429+0000 7f9918fe9700 1 --2- 172.21.15.32:0/1157567493 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9914108410 0x7f9914199940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:43:56.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:56.429+0000 7f99137fe700 1 -- 172.21.15.32:0/1157567493 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f99141023a0 msgr2=0x7f991419f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:43:56.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:56.429+0000 7f99137fe700 1 --2- 172.21.15.32:0/1157567493 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f99141023a0 0x7f991419f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:56.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:56.429+0000 7f99137fe700 1 -- 172.21.15.32:0/1157567493 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9914108410 msgr2=0x7f9914199940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:43:56.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:56.429+0000 7f99137fe700 1 --2- 172.21.15.32:0/1157567493 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9914108410 0x7f9914199940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:56.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:56.429+0000 7f99137fe700 1 -- 172.21.15.32:0/1157567493 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f991419a0a0 con 0x7f9914102d60 2024-05-03T18:43:56.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:56.429+0000 7f9913fff700 1 --2- 172.21.15.32:0/1157567493 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f99141023a0 0x7f991419f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T18:43:56.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:56.429+0000 7f99137fe700 1 --2- 172.21.15.32:0/1157567493 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9914102d60 0x7f991419fcc0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f9908000c00 tx=0x7f9908009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:43:56.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:56.429+0000 7f99117fa700 1 -- 172.21.15.32:0/1157567493 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9908027070 con 0x7f9914102d60 2024-05-03T18:43:56.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:56.429+0000 7f99117fa700 1 -- 172.21.15.32:0/1157567493 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9908022470 con 0x7f9914102d60 2024-05-03T18:43:56.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:56.429+0000 7f991aa4c700 1 -- 172.21.15.32:0/1157567493 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f991419a330 con 0x7f9914102d60 2024-05-03T18:43:56.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:56.429+0000 7f99117fa700 1 -- 172.21.15.32:0/1157567493 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9908017650 con 0x7f9914102d60 2024-05-03T18:43:56.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:56.429+0000 7f991aa4c700 1 -- 172.21.15.32:0/1157567493 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99141a6650 con 0x7f9914102d60 2024-05-03T18:43:56.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:56.430+0000 7f991aa4c700 1 -- 172.21.15.32:0/1157567493 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f991406a190 con 0x7f9914102d60 2024-05-03T18:43:56.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:56.432+0000 7f99117fa700 1 -- 172.21.15.32:0/1157567493 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f990802a070 con 0x7f9914102d60 2024-05-03T18:43:56.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:56.432+0000 7f99117fa700 1 --2- 172.21.15.32:0/1157567493 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f98fc0b2900 0x7f98fc0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:43:56.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:56.432+0000 7f9913fff700 1 --2- 172.21.15.32:0/1157567493 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f98fc0b2900 0x7f98fc0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:43:56.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:56.432+0000 7f99117fa700 1 -- 172.21.15.32:0/1157567493 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f99080e2a50 con 0x7f9914102d60 2024-05-03T18:43:56.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:56.433+0000 7f9913fff700 1 --2- 172.21.15.32:0/1157567493 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f98fc0b2900 0x7f98fc0b4dc0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f9904005950 tx=0x7f990400a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:43:56.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:56.437+0000 7f99117fa700 1 -- 172.21.15.32:0/1157567493 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f9908013070 con 0x7f9914102d60 2024-05-03T18:43:56.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:56.649+0000 7f991aa4c700 1 -- 172.21.15.32:0/1157567493 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f99141a6820 con 0x7f9914102d60 2024-05-03T18:43:56.650 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:56.649+0000 7f99117fa700 1 -- 172.21.15.32:0/1157567493 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v71) ==== 93+0+10 (secure 0 0 0) 0x7f99080aaca0 con 0x7f9914102d60 2024-05-03T18:43:56.651 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2024-05-03T18:43:56.652 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:56.651+0000 7f991aa4c700 1 -- 172.21.15.32:0/1157567493 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f98fc0b2900 msgr2=0x7f98fc0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:43:56.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:56.651+0000 7f991aa4c700 1 --2- 172.21.15.32:0/1157567493 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f98fc0b2900 0x7f98fc0b4dc0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f9904005950 tx=0x7f990400a300 comp rx=0 tx=0).stop 2024-05-03T18:43:56.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:56.651+0000 7f991aa4c700 1 -- 172.21.15.32:0/1157567493 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9914102d60 msgr2=0x7f991419fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:43:56.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:56.651+0000 7f991aa4c700 1 --2- 172.21.15.32:0/1157567493 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9914102d60 0x7f991419fcc0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f9908000c00 tx=0x7f9908009580 comp rx=0 tx=0).stop 2024-05-03T18:43:56.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:56.652+0000 7f991aa4c700 1 -- 172.21.15.32:0/1157567493 shutdown_connections 2024-05-03T18:43:56.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:56.652+0000 7f991aa4c700 1 --2- 172.21.15.32:0/1157567493 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f98fc0b2900 0x7f98fc0b4dc0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:56.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:56.652+0000 7f991aa4c700 1 --2- 172.21.15.32:0/1157567493 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9914108410 0x7f9914199940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:56.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:56.652+0000 7f991aa4c700 1 --2- 172.21.15.32:0/1157567493 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f99141023a0 0x7f991419f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:56.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:56.652+0000 7f991aa4c700 1 --2- 172.21.15.32:0/1157567493 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9914102d60 0x7f991419fcc0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:56.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:56.652+0000 7f991aa4c700 1 -- 172.21.15.32:0/1157567493 >> 172.21.15.32:0/1157567493 conn(0x7f9914079350 msgr2=0x7f9914101660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:43:56.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:56.652+0000 7f991aa4c700 1 -- 172.21.15.32:0/1157567493 shutdown_connections 2024-05-03T18:43:56.653 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:43:56.652+0000 7f991aa4c700 1 -- 172.21.15.32:0/1157567493 wait complete. 2024-05-03T18:43:57.300 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:57 smithi045 ceph-mon[33299]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:43:57.300 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:57 smithi045 ceph-mon[33299]: from='client.? 172.21.15.32:0/1157567493' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-03T18:43:57.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:57 smithi113 ceph-mon[33540]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:43:57.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:57 smithi113 ceph-mon[33540]: from='client.? 172.21.15.32:0/1157567493' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-03T18:43:57.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:57 smithi032 ceph-mon[29763]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:43:57.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:57 smithi032 ceph-mon[29763]: from='client.? 172.21.15.32:0/1157567493' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-03T18:43:57.374 INFO:tasks.cephadm:Adding mds.a on smithi032 2024-05-03T18:43:57.374 INFO:tasks.cephadm:Adding mds.d on smithi032 2024-05-03T18:43:57.374 INFO:tasks.cephadm:Adding mds.g on smithi032 2024-05-03T18:43:57.374 INFO:tasks.cephadm:Adding mds.j on smithi032 2024-05-03T18:43:57.374 INFO:tasks.cephadm:Adding mds.b on smithi045 2024-05-03T18:43:57.374 INFO:tasks.cephadm:Adding mds.e on smithi045 2024-05-03T18:43:57.374 INFO:tasks.cephadm:Adding mds.h on smithi045 2024-05-03T18:43:57.374 INFO:tasks.cephadm:Adding mds.k on smithi045 2024-05-03T18:43:57.375 INFO:tasks.cephadm:Adding mds.c on smithi113 2024-05-03T18:43:57.375 INFO:tasks.cephadm:Adding mds.f on smithi113 2024-05-03T18:43:57.375 INFO:tasks.cephadm:Adding mds.i on smithi113 2024-05-03T18:43:57.375 INFO:tasks.cephadm:Adding mds.l on smithi113 2024-05-03T18:43:57.375 DEBUG:teuthology.orchestra.run.smithi113:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f500455e-097b-11ef-bc97-c7b262605968 -- ceph orch apply mds all '12;smithi032=a;smithi032=d;smithi032=g;smithi032=j;smithi045=b;smithi045=e;smithi045=h;smithi045=k;smithi113=c;smithi113=f;smithi113=i;smithi113=l' 2024-05-03T18:43:57.626 INFO:teuthology.orchestra.run.smithi113.stderr:Inferring config /var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/mon.c/config 2024-05-03T18:43:59.206 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:43:59.202+0000 7fa5dff74700 1 -- 172.21.15.113:0/475652133 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa5d81035f0 msgr2=0x7fa5d8103a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:43:59.206 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:43:59.202+0000 7fa5dff74700 1 --2- 172.21.15.113:0/475652133 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa5d81035f0 0x7fa5d8103a50 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa5c8009990 tx=0x7fa5c8009ca0 comp rx=0 tx=0).stop 2024-05-03T18:43:59.206 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:43:59.203+0000 7fa5dff74700 1 -- 172.21.15.113:0/475652133 shutdown_connections 2024-05-03T18:43:59.206 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:43:59.203+0000 7fa5dff74700 1 --2- 172.21.15.113:0/475652133 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5d8103f90 0x7fa5d810a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:59.206 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:43:59.203+0000 7fa5dff74700 1 --2- 172.21.15.113:0/475652133 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5d81023f0 0x7fa5d8102810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:59.206 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:43:59.203+0000 7fa5dff74700 1 --2- 172.21.15.113:0/475652133 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa5d81035f0 0x7fa5d8103a50 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:59.206 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:43:59.203+0000 7fa5dff74700 1 -- 172.21.15.113:0/475652133 >> 172.21.15.113:0/475652133 conn(0x7fa5d80fd990 msgr2=0x7fa5d80ffdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:43:59.206 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:43:59.203+0000 7fa5dff74700 1 -- 172.21.15.113:0/475652133 shutdown_connections 2024-05-03T18:43:59.206 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:43:59.203+0000 7fa5dff74700 1 -- 172.21.15.113:0/475652133 wait complete. 2024-05-03T18:43:59.206 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:43:59.203+0000 7fa5dff74700 1 Processor -- start 2024-05-03T18:43:59.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:43:59.203+0000 7fa5dff74700 1 -- start start 2024-05-03T18:43:59.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:43:59.203+0000 7fa5dff74700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5d81023f0 0x7fa5d8199610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:43:59.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:43:59.203+0000 7fa5dff74700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5d81035f0 0x7fa5d8199b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:43:59.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:43:59.203+0000 7fa5dff74700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa5d8103f90 0x7fa5d81a0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:43:59.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:43:59.203+0000 7fa5dff74700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa5d810de80 con 0x7fa5d81023f0 2024-05-03T18:43:59.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:43:59.203+0000 7fa5dff74700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa5d810dd00 con 0x7fa5d81035f0 2024-05-03T18:43:59.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:43:59.203+0000 7fa5dff74700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fa5d810e000 con 0x7fa5d8103f90 2024-05-03T18:43:59.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:43:59.203+0000 7fa5de511700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa5d8103f90 0x7fa5d81a0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:43:59.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:43:59.204+0000 7fa5de511700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa5d8103f90 0x7fa5d81a0bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:54520/0 (socket says 172.21.15.113:54520) 2024-05-03T18:43:59.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:43:59.204+0000 7fa5de511700 1 -- 172.21.15.113:0/878051029 learned_addr learned my addr 172.21.15.113:0/878051029 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-05-03T18:43:59.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:43:59.204+0000 7fa5ddd10700 1 --2- 172.21.15.113:0/878051029 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5d81023f0 0x7fa5d8199610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:43:59.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:43:59.204+0000 7fa5dd50f700 1 --2- 172.21.15.113:0/878051029 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5d81035f0 0x7fa5d8199b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:43:59.207 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:43:59.204+0000 7fa5de511700 1 -- 172.21.15.113:0/878051029 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5d81035f0 msgr2=0x7fa5d8199b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:43:59.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:43:59.204+0000 7fa5de511700 1 --2- 172.21.15.113:0/878051029 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5d81035f0 0x7fa5d8199b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:59.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:43:59.204+0000 7fa5de511700 1 -- 172.21.15.113:0/878051029 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5d81023f0 msgr2=0x7fa5d8199610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:43:59.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:43:59.204+0000 7fa5de511700 1 --2- 172.21.15.113:0/878051029 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5d81023f0 0x7fa5d8199610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:59.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:43:59.204+0000 7fa5de511700 1 -- 172.21.15.113:0/878051029 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5d81a1270 con 0x7fa5d8103f90 2024-05-03T18:43:59.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:43:59.204+0000 7fa5de511700 1 --2- 172.21.15.113:0/878051029 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa5d8103f90 0x7fa5d81a0bd0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa5d0007f10 tx=0x7fa5d000d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:43:59.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:43:59.204+0000 7fa5ceffd700 1 -- 172.21.15.113:0/878051029 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5d0004bb0 con 0x7fa5d8103f90 2024-05-03T18:43:59.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:43:59.204+0000 7fa5dff74700 1 -- 172.21.15.113:0/878051029 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5d81a1560 con 0x7fa5d8103f90 2024-05-03T18:43:59.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:43:59.204+0000 7fa5ceffd700 1 -- 172.21.15.113:0/878051029 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa5d0004d50 con 0x7fa5d8103f90 2024-05-03T18:43:59.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:43:59.204+0000 7fa5ceffd700 1 -- 172.21.15.113:0/878051029 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5d00058e0 con 0x7fa5d8103f90 2024-05-03T18:43:59.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:43:59.204+0000 7fa5dff74700 1 -- 172.21.15.113:0/878051029 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5d81a1af0 con 0x7fa5d8103f90 2024-05-03T18:43:59.208 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:43:59.205+0000 7fa5dff74700 1 -- 172.21.15.113:0/878051029 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5d806a190 con 0x7fa5d8103f90 2024-05-03T18:43:59.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:43:59.207+0000 7fa5ceffd700 1 -- 172.21.15.113:0/878051029 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fa5d001a020 con 0x7fa5d8103f90 2024-05-03T18:43:59.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:43:59.208+0000 7fa5ceffd700 1 --2- 172.21.15.113:0/878051029 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fa5c40b2880 0x7fa5c40b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:43:59.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:43:59.208+0000 7fa5ceffd700 1 -- 172.21.15.113:0/878051029 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7fa5d00e1aa0 con 0x7fa5d8103f90 2024-05-03T18:43:59.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:43:59.208+0000 7fa5ddd10700 1 --2- 172.21.15.113:0/878051029 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fa5c40b2880 0x7fa5c40b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:43:59.209 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:43:59.208+0000 7fa5ddd10700 1 --2- 172.21.15.113:0/878051029 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fa5c40b2880 0x7fa5c40b4d40 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa5d8103450 tx=0x7fa5d400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:43:59.212 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:43:59.211+0000 7fa5ceffd700 1 -- 172.21.15.113:0/878051029 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fa5d0010070 con 0x7fa5d8103f90 2024-05-03T18:43:59.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:43:59 smithi113 ceph-mon[33540]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:43:59.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:43:59 smithi032 ceph-mon[29763]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:43:59.426 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:43:59.425+0000 7fa5dff74700 1 -- 172.21.15.113:0/878051029 --> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi032=a;smithi032=d;smithi032=g;smithi032=j;smithi045=b;smithi045=e;smithi045=h;smithi045=k;smithi113=c;smithi113=f;smithi113=i;smithi113=l", "target": ["mon-mgr", ""]}) -- 0x7fa5d806c460 con 0x7fa5c40b2880 2024-05-03T18:43:59.434 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:43:59.433+0000 7fa5ceffd700 1 -- 172.21.15.113:0/878051029 <== mgr.14152 v2:172.21.15.32:6800/1796547904 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7fa5d806c460 con 0x7fa5c40b2880 2024-05-03T18:43:59.434 INFO:teuthology.orchestra.run.smithi113.stdout:Scheduled mds.all update... 2024-05-03T18:43:59.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:43:59.435+0000 7fa5dff74700 1 -- 172.21.15.113:0/878051029 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fa5c40b2880 msgr2=0x7fa5c40b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:43:59.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:43:59.435+0000 7fa5dff74700 1 --2- 172.21.15.113:0/878051029 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fa5c40b2880 0x7fa5c40b4d40 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa5d8103450 tx=0x7fa5d400b410 comp rx=0 tx=0).stop 2024-05-03T18:43:59.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:43:59.435+0000 7fa5dff74700 1 -- 172.21.15.113:0/878051029 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa5d8103f90 msgr2=0x7fa5d81a0bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:43:59.436 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:43:59.435+0000 7fa5dff74700 1 --2- 172.21.15.113:0/878051029 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa5d8103f90 0x7fa5d81a0bd0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa5d0007f10 tx=0x7fa5d000d3b0 comp rx=0 tx=0).stop 2024-05-03T18:43:59.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:43:59.435+0000 7fa5dff74700 1 -- 172.21.15.113:0/878051029 shutdown_connections 2024-05-03T18:43:59.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:43:59.435+0000 7fa5dff74700 1 --2- 172.21.15.113:0/878051029 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fa5c40b2880 0x7fa5c40b4d40 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:59.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:43:59.435+0000 7fa5dff74700 1 --2- 172.21.15.113:0/878051029 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa5d8103f90 0x7fa5d81a0bd0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:59.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:43:59.435+0000 7fa5dff74700 1 --2- 172.21.15.113:0/878051029 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5d81023f0 0x7fa5d8199610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:59.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:43:59.435+0000 7fa5dff74700 1 --2- 172.21.15.113:0/878051029 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5d81035f0 0x7fa5d8199b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:43:59.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:43:59.435+0000 7fa5dff74700 1 -- 172.21.15.113:0/878051029 >> 172.21.15.113:0/878051029 conn(0x7fa5d80fd990 msgr2=0x7fa5d806bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:43:59.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:43:59.435+0000 7fa5dff74700 1 -- 172.21.15.113:0/878051029 shutdown_connections 2024-05-03T18:43:59.437 INFO:teuthology.orchestra.run.smithi113.stderr:2024-05-03T18:43:59.435+0000 7fa5dff74700 1 -- 172.21.15.113:0/878051029 wait complete. 2024-05-03T18:43:59.497 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:43:59 smithi045 ceph-mon[33299]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:44:00.050 DEBUG:teuthology.orchestra.run.smithi032:mds.a> sudo journalctl -f -n 0 -u ceph-f500455e-097b-11ef-bc97-c7b262605968@mds.a.service 2024-05-03T18:44:00.053 DEBUG:teuthology.orchestra.run.smithi032:mds.d> sudo journalctl -f -n 0 -u ceph-f500455e-097b-11ef-bc97-c7b262605968@mds.d.service 2024-05-03T18:44:00.097 DEBUG:teuthology.orchestra.run.smithi032:mds.g> sudo journalctl -f -n 0 -u ceph-f500455e-097b-11ef-bc97-c7b262605968@mds.g.service 2024-05-03T18:44:00.142 DEBUG:teuthology.orchestra.run.smithi032:mds.j> sudo journalctl -f -n 0 -u ceph-f500455e-097b-11ef-bc97-c7b262605968@mds.j.service 2024-05-03T18:44:00.184 DEBUG:teuthology.orchestra.run.smithi045:mds.b> sudo journalctl -f -n 0 -u ceph-f500455e-097b-11ef-bc97-c7b262605968@mds.b.service 2024-05-03T18:44:00.187 DEBUG:teuthology.orchestra.run.smithi045:mds.e> sudo journalctl -f -n 0 -u ceph-f500455e-097b-11ef-bc97-c7b262605968@mds.e.service 2024-05-03T18:44:00.230 DEBUG:teuthology.orchestra.run.smithi045:mds.h> sudo journalctl -f -n 0 -u ceph-f500455e-097b-11ef-bc97-c7b262605968@mds.h.service 2024-05-03T18:44:00.273 DEBUG:teuthology.orchestra.run.smithi045:mds.k> sudo journalctl -f -n 0 -u ceph-f500455e-097b-11ef-bc97-c7b262605968@mds.k.service 2024-05-03T18:44:00.316 DEBUG:teuthology.orchestra.run.smithi113:mds.c> sudo journalctl -f -n 0 -u ceph-f500455e-097b-11ef-bc97-c7b262605968@mds.c.service 2024-05-03T18:44:00.318 DEBUG:teuthology.orchestra.run.smithi113:mds.f> sudo journalctl -f -n 0 -u ceph-f500455e-097b-11ef-bc97-c7b262605968@mds.f.service 2024-05-03T18:44:00.361 DEBUG:teuthology.orchestra.run.smithi113:mds.i> sudo journalctl -f -n 0 -u ceph-f500455e-097b-11ef-bc97-c7b262605968@mds.i.service 2024-05-03T18:44:00.404 DEBUG:teuthology.orchestra.run.smithi113:mds.l> sudo journalctl -f -n 0 -u ceph-f500455e-097b-11ef-bc97-c7b262605968@mds.l.service 2024-05-03T18:44:00.446 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-05-03T18:44:00.446 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid f500455e-097b-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-03T18:44:00.497 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:00 smithi045 ceph-mon[33299]: from='client.24418 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi032=a;smithi032=d;smithi032=g;smithi032=j;smithi045=b;smithi045=e;smithi045=h;smithi045=k;smithi113=c;smithi113=f;smithi113=i;smithi113=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:44:00.497 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:00 smithi045 ceph-mon[33299]: Saving service mds.all spec with placement smithi032=a;smithi032=d;smithi032=g;smithi032=j;smithi045=b;smithi045=e;smithi045=h;smithi045=k;smithi113=c;smithi113=f;smithi113=i;smithi113=l;count:12 2024-05-03T18:44:00.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:00 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:00.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:00 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:44:00.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:00 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:44:00.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:00 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:44:00.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:00 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:00.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:00 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T18:44:00.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:00 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T18:44:00.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:00 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:44:00.689 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:44:00.696 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:00 smithi113 ceph-mon[33540]: from='client.24418 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi032=a;smithi032=d;smithi032=g;smithi032=j;smithi045=b;smithi045=e;smithi045=h;smithi045=k;smithi113=c;smithi113=f;smithi113=i;smithi113=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:44:00.696 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:00 smithi113 ceph-mon[33540]: Saving service mds.all spec with placement smithi032=a;smithi032=d;smithi032=g;smithi032=j;smithi045=b;smithi045=e;smithi045=h;smithi045=k;smithi113=c;smithi113=f;smithi113=i;smithi113=l;count:12 2024-05-03T18:44:00.696 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:00 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:00.696 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:00 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:44:00.696 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:00 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:44:00.696 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:00 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:44:00.696 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:00 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:00.697 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:00 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T18:44:00.697 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:00 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T18:44:00.697 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:00 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:44:00.811 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:00 smithi032 ceph-mon[29763]: from='client.24418 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi032=a;smithi032=d;smithi032=g;smithi032=j;smithi045=b;smithi045=e;smithi045=h;smithi045=k;smithi113=c;smithi113=f;smithi113=i;smithi113=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:44:00.812 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:00 smithi032 ceph-mon[29763]: Saving service mds.all spec with placement smithi032=a;smithi032=d;smithi032=g;smithi032=j;smithi045=b;smithi045=e;smithi045=h;smithi045=k;smithi113=c;smithi113=f;smithi113=i;smithi113=l;count:12 2024-05-03T18:44:00.812 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:00 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:00.812 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:00 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:44:00.812 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:00 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:44:00.812 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:00 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:44:00.812 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:00 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:00.812 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:00 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T18:44:00.812 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:00 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T18:44:00.812 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:00 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:44:01.634 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:01 smithi113 ceph-mon[33540]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:44:01.634 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:01 smithi113 ceph-mon[33540]: Deploying daemon mds.l on smithi113 2024-05-03T18:44:01.747 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:01 smithi045 ceph-mon[33299]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:44:01.747 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:01 smithi045 ceph-mon[33299]: Deploying daemon mds.l on smithi113 2024-05-03T18:44:01.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:01 smithi032 ceph-mon[29763]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:44:01.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:01 smithi032 ceph-mon[29763]: Deploying daemon mds.l on smithi113 2024-05-03T18:44:02.301 INFO:journalctl@ceph.mds.l.smithi113.stdout:May 03 18:44:02 smithi113 systemd[1]: Starting Ceph mds.l for f500455e-097b-11ef-bc97-c7b262605968... 2024-05-03T18:44:02.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:02.368+0000 7fd7269fd700 1 -- 172.21.15.32:0/803383317 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd72006c190 msgr2=0x7fd72006c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:44:02.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:02.368+0000 7fd7269fd700 1 --2- 172.21.15.32:0/803383317 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd72006c190 0x7fd72006c610 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd708009990 tx=0x7fd708009ca0 comp rx=0 tx=0).stop 2024-05-03T18:44:02.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:02.368+0000 7fd7269fd700 1 -- 172.21.15.32:0/803383317 shutdown_connections 2024-05-03T18:44:02.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:02.368+0000 7fd7269fd700 1 --2- 172.21.15.32:0/803383317 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd720105f10 0x7fd720106370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:02.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:02.368+0000 7fd7269fd700 1 --2- 172.21.15.32:0/803383317 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd720108400 0x7fd72006bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:02.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:02.368+0000 7fd7269fd700 1 --2- 172.21.15.32:0/803383317 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd72006c190 0x7fd72006c610 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:02.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:02.368+0000 7fd7269fd700 1 -- 172.21.15.32:0/803383317 >> 172.21.15.32:0/803383317 conn(0x7fd720079440 msgr2=0x7fd720079850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:44:02.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:02.368+0000 7fd7269fd700 1 -- 172.21.15.32:0/803383317 shutdown_connections 2024-05-03T18:44:02.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:02.368+0000 7fd7269fd700 1 -- 172.21.15.32:0/803383317 wait complete. 2024-05-03T18:44:02.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:02.369+0000 7fd7269fd700 1 Processor -- start 2024-05-03T18:44:02.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:02.369+0000 7fd7269fd700 1 -- start start 2024-05-03T18:44:02.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:02.369+0000 7fd7269fd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd72006c190 0x7fd7201a0130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:02.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:02.369+0000 7fd7269fd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd720105f10 0x7fd7201a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:02.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:02.369+0000 7fd7269fd700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd720108400 0x7fd7201a4a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:02.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:02.369+0000 7fd7269fd700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd720119c80 con 0x7fd720105f10 2024-05-03T18:44:02.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:02.369+0000 7fd7269fd700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd720119b00 con 0x7fd72006c190 2024-05-03T18:44:02.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:02.370+0000 7fd7269fd700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fd720119e00 con 0x7fd720108400 2024-05-03T18:44:02.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:02.370+0000 7fd71f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd720105f10 0x7fd7201a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:44:02.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:02.370+0000 7fd71f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd720105f10 0x7fd7201a0670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38696/0 (socket says 172.21.15.32:38696) 2024-05-03T18:44:02.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:02.370+0000 7fd724f9a700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd720108400 0x7fd7201a4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:44:02.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:02.370+0000 7fd71f7fe700 1 -- 172.21.15.32:0/652609702 learned_addr learned my addr 172.21.15.32:0/652609702 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:44:02.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:02.370+0000 7fd71ffff700 1 --2- 172.21.15.32:0/652609702 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd72006c190 0x7fd7201a0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:44:02.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:02.370+0000 7fd71f7fe700 1 -- 172.21.15.32:0/652609702 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd720108400 msgr2=0x7fd7201a4a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:44:02.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:02.370+0000 7fd71f7fe700 1 --2- 172.21.15.32:0/652609702 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd720108400 0x7fd7201a4a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:02.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:02.370+0000 7fd71f7fe700 1 -- 172.21.15.32:0/652609702 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd72006c190 msgr2=0x7fd7201a0130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:44:02.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:02.370+0000 7fd71f7fe700 1 --2- 172.21.15.32:0/652609702 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd72006c190 0x7fd7201a0130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:02.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:02.370+0000 7fd71f7fe700 1 -- 172.21.15.32:0/652609702 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7201a5140 con 0x7fd720105f10 2024-05-03T18:44:02.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:02.370+0000 7fd71f7fe700 1 --2- 172.21.15.32:0/652609702 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd720105f10 0x7fd7201a0670 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd7080057c0 tx=0x7fd708005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:44:02.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:02.370+0000 7fd71d7fa700 1 -- 172.21.15.32:0/652609702 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd708027070 con 0x7fd720105f10 2024-05-03T18:44:02.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:02.370+0000 7fd7269fd700 1 -- 172.21.15.32:0/652609702 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7201a53d0 con 0x7fd720105f10 2024-05-03T18:44:02.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:02.370+0000 7fd71d7fa700 1 -- 172.21.15.32:0/652609702 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd70800fb20 con 0x7fd720105f10 2024-05-03T18:44:02.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:02.370+0000 7fd71d7fa700 1 -- 172.21.15.32:0/652609702 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd708022b80 con 0x7fd720105f10 2024-05-03T18:44:02.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:02.370+0000 7fd7269fd700 1 -- 172.21.15.32:0/652609702 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7201accb0 con 0x7fd720105f10 2024-05-03T18:44:02.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:02.371+0000 7fd7269fd700 1 -- 172.21.15.32:0/652609702 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd72006a190 con 0x7fd720105f10 2024-05-03T18:44:02.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:02.373+0000 7fd71d7fa700 1 -- 172.21.15.32:0/652609702 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fd70802a070 con 0x7fd720105f10 2024-05-03T18:44:02.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:02.373+0000 7fd71d7fa700 1 --2- 172.21.15.32:0/652609702 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fd70c0b2880 0x7fd70c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:02.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:02.373+0000 7fd71d7fa700 1 -- 172.21.15.32:0/652609702 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7fd7080e2d40 con 0x7fd720105f10 2024-05-03T18:44:02.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:02.373+0000 7fd71ffff700 1 --2- 172.21.15.32:0/652609702 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fd70c0b2880 0x7fd70c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:44:02.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:02.374+0000 7fd71ffff700 1 --2- 172.21.15.32:0/652609702 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fd70c0b2880 0x7fd70c0b4d40 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fd710005950 tx=0x7fd71000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:44:02.379 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:02.378+0000 7fd71d7fa700 1 -- 172.21.15.32:0/652609702 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fd708013070 con 0x7fd720105f10 2024-05-03T18:44:02.625 INFO:journalctl@ceph.mds.l.smithi113.stdout:May 03 18:44:02 smithi113 podman[64272]: 2024-05-03 18:44:02.22004459 +0000 UTC m=+0.017909870 image pull ee4d846202496ba6096593e8e8becc0700a5eb7ebd95536031187f5f337d4012 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd 2024-05-03T18:44:02.625 INFO:journalctl@ceph.mds.l.smithi113.stdout:May 03 18:44:02 smithi113 podman[64272]: 2024-05-03 18:44:02.32204379 +0000 UTC m=+0.119909069 container create 4f9cf271e240b6c89983e04173d825209567bbf7353d4a5d3c75b463157a1c15 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-mds-l, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240430, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-05-03T18:44:02.625 INFO:journalctl@ceph.mds.l.smithi113.stdout:May 03 18:44:02 smithi113 podman[64272]: 2024-05-03 18:44:02.533032186 +0000 UTC m=+0.330897467 container init 4f9cf271e240b6c89983e04173d825209567bbf7353d4a5d3c75b463157a1c15 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-mds-l, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240430, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-05-03T18:44:02.625 INFO:journalctl@ceph.mds.l.smithi113.stdout:May 03 18:44:02 smithi113 podman[64272]: 2024-05-03 18:44:02.535884668 +0000 UTC m=+0.333749949 container start 4f9cf271e240b6c89983e04173d825209567bbf7353d4a5d3c75b463157a1c15 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-mds-l, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240430, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True) 2024-05-03T18:44:02.625 INFO:journalctl@ceph.mds.l.smithi113.stdout:May 03 18:44:02 smithi113 ceph-mds[64291]: -- 172.21.15.113:0/2614490225 <== mon.1 v2:172.21.15.113:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55672d47da40 con 0x55672d450800 2024-05-03T18:44:02.625 INFO:journalctl@ceph.mds.l.smithi113.stdout:May 03 18:44:02 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-mds-l[64287]: starting mds.l at 2024-05-03T18:44:02.625 INFO:journalctl@ceph.mds.l.smithi113.stdout:May 03 18:44:02 smithi113 bash[64272]: 4f9cf271e240b6c89983e04173d825209567bbf7353d4a5d3c75b463157a1c15 2024-05-03T18:44:02.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:02.643+0000 7fd7269fd700 1 -- 172.21.15.32:0/652609702 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd7201ace80 con 0x7fd720105f10 2024-05-03T18:44:02.644 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:02.643+0000 7fd71d7fa700 1 -- 172.21.15.32:0/652609702 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1396 (secure 0 0 0) 0x7fd7080aaf90 con 0x7fd720105f10 2024-05-03T18:44:02.644 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-03T18:44:02.644 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6833/3323558791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":3323558791},{"type":"v1","addr":"172.21.15.113:6833","nonce":3323558791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-05-03T18:44:02.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:02.645+0000 7fd7269fd700 1 -- 172.21.15.32:0/652609702 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fd70c0b2880 msgr2=0x7fd70c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:44:02.646 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:02.645+0000 7fd7269fd700 1 --2- 172.21.15.32:0/652609702 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fd70c0b2880 0x7fd70c0b4d40 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fd710005950 tx=0x7fd71000b410 comp rx=0 tx=0).stop 2024-05-03T18:44:02.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:02.645+0000 7fd7269fd700 1 -- 172.21.15.32:0/652609702 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd720105f10 msgr2=0x7fd7201a0670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:44:02.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:02.645+0000 7fd7269fd700 1 --2- 172.21.15.32:0/652609702 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd720105f10 0x7fd7201a0670 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd7080057c0 tx=0x7fd708005ef0 comp rx=0 tx=0).stop 2024-05-03T18:44:02.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:02.645+0000 7fd7269fd700 1 -- 172.21.15.32:0/652609702 shutdown_connections 2024-05-03T18:44:02.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:02.645+0000 7fd7269fd700 1 --2- 172.21.15.32:0/652609702 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fd70c0b2880 0x7fd70c0b4d40 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:02.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:02.645+0000 7fd7269fd700 1 --2- 172.21.15.32:0/652609702 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd720108400 0x7fd7201a4a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:02.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:02.645+0000 7fd7269fd700 1 --2- 172.21.15.32:0/652609702 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd72006c190 0x7fd7201a0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:02.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:02.645+0000 7fd7269fd700 1 --2- 172.21.15.32:0/652609702 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd720105f10 0x7fd7201a0670 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:02.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:02.645+0000 7fd7269fd700 1 -- 172.21.15.32:0/652609702 >> 172.21.15.32:0/652609702 conn(0x7fd720079440 msgr2=0x7fd720104ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:44:02.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:02.645+0000 7fd7269fd700 1 -- 172.21.15.32:0/652609702 shutdown_connections 2024-05-03T18:44:02.647 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:02.645+0000 7fd7269fd700 1 -- 172.21.15.32:0/652609702 wait complete. 2024-05-03T18:44:02.647 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 2 2024-05-03T18:44:03.051 INFO:journalctl@ceph.mds.l.smithi113.stdout:May 03 18:44:02 smithi113 systemd[1]: Started Ceph mds.l for f500455e-097b-11ef-bc97-c7b262605968. 2024-05-03T18:44:03.660 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:03 smithi045 ceph-mon[33299]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:44:03.660 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:03 smithi045 ceph-mon[33299]: mds.? [v2:172.21.15.113:6832/3323558791,v1:172.21.15.113:6833/3323558791] up:boot 2024-05-03T18:44:03.660 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:03 smithi045 ceph-mon[33299]: fsmap 1 up:standby 2024-05-03T18:44:03.661 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:03 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-03T18:44:03.661 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:03 smithi045 ceph-mon[33299]: from='client.? 172.21.15.32:0/652609702' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:44:03.661 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:03 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:03.661 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:03 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:03.661 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:03 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:03.661 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:03 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T18:44:03.661 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:03 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T18:44:03.661 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:03 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:44:03.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:03 smithi032 ceph-mon[29763]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:44:03.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:03 smithi032 ceph-mon[29763]: mds.? [v2:172.21.15.113:6832/3323558791,v1:172.21.15.113:6833/3323558791] up:boot 2024-05-03T18:44:03.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:03 smithi032 ceph-mon[29763]: fsmap 1 up:standby 2024-05-03T18:44:03.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:03 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-03T18:44:03.819 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:03 smithi032 ceph-mon[29763]: from='client.? 172.21.15.32:0/652609702' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:44:03.819 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:03 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:03.819 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:03 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:03.819 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:03 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:03.819 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:03 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T18:44:03.819 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:03 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T18:44:03.819 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:03 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:44:04.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:03 smithi113 ceph-mon[33540]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:44:04.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:03 smithi113 ceph-mon[33540]: mds.? [v2:172.21.15.113:6832/3323558791,v1:172.21.15.113:6833/3323558791] up:boot 2024-05-03T18:44:04.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:03 smithi113 ceph-mon[33540]: fsmap 1 up:standby 2024-05-03T18:44:04.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:03 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-03T18:44:04.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:03 smithi113 ceph-mon[33540]: from='client.? 172.21.15.32:0/652609702' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:44:04.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:03 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:04.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:03 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:04.052 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:03 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:04.052 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:03 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T18:44:04.052 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:03 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T18:44:04.052 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:03 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:44:04.720 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:04 smithi045 ceph-mon[33299]: Deploying daemon mds.k on smithi045 2024-05-03T18:44:04.998 INFO:journalctl@ceph.mds.k.smithi045.stdout:May 03 18:44:04 smithi045 systemd[1]: Starting Ceph mds.k for f500455e-097b-11ef-bc97-c7b262605968... 2024-05-03T18:44:05.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:04 smithi113 ceph-mon[33540]: Deploying daemon mds.k on smithi045 2024-05-03T18:44:05.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:04 smithi032 ceph-mon[29763]: Deploying daemon mds.k on smithi045 2024-05-03T18:44:05.242 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid f500455e-097b-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-03T18:44:05.418 INFO:journalctl@ceph.mds.k.smithi045.stdout:May 03 18:44:05 smithi045 podman[62834]: 2024-05-03 18:44:05.12403889 +0000 UTC m=+0.106716046 container create 01184527005bcc1c52bab3de064db9e891de3a51c00c75bcaf192db2f689a34e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-mds-k, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240430, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2) 2024-05-03T18:44:05.418 INFO:journalctl@ceph.mds.k.smithi045.stdout:May 03 18:44:05 smithi045 podman[62834]: 2024-05-03 18:44:05.035560837 +0000 UTC m=+0.018237948 image pull ee4d846202496ba6096593e8e8becc0700a5eb7ebd95536031187f5f337d4012 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd 2024-05-03T18:44:05.419 INFO:journalctl@ceph.mds.k.smithi045.stdout:May 03 18:44:05 smithi045 podman[62834]: 2024-05-03 18:44:05.325942728 +0000 UTC m=+0.308619838 container init 01184527005bcc1c52bab3de064db9e891de3a51c00c75bcaf192db2f689a34e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-mds-k, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240430, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-05-03T18:44:05.419 INFO:journalctl@ceph.mds.k.smithi045.stdout:May 03 18:44:05 smithi045 podman[62834]: 2024-05-03 18:44:05.328653156 +0000 UTC m=+0.311330268 container start 01184527005bcc1c52bab3de064db9e891de3a51c00c75bcaf192db2f689a34e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-mds-k, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240430, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe) 2024-05-03T18:44:05.419 INFO:journalctl@ceph.mds.k.smithi045.stdout:May 03 18:44:05 smithi045 ceph-mds[62853]: -- 172.21.15.45:0/155402547 <== mon.2 v2:172.21.15.45:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55994c4dda40 con 0x55994c4b0800 2024-05-03T18:44:05.419 INFO:journalctl@ceph.mds.k.smithi045.stdout:May 03 18:44:05 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-mds-k[62849]: starting mds.k at 2024-05-03T18:44:05.419 INFO:journalctl@ceph.mds.k.smithi045.stdout:May 03 18:44:05 smithi045 bash[62834]: 01184527005bcc1c52bab3de064db9e891de3a51c00c75bcaf192db2f689a34e 2024-05-03T18:44:05.501 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:44:05.748 INFO:journalctl@ceph.mds.k.smithi045.stdout:May 03 18:44:05 smithi045 systemd[1]: Started Ceph mds.k for f500455e-097b-11ef-bc97-c7b262605968. 2024-05-03T18:44:05.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:05 smithi045 ceph-mon[33299]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:44:05.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:05 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:05.749 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:05 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:05.749 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:05 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:05.749 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:05 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T18:44:05.749 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:05 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T18:44:05.749 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:05 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:44:05.749 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:05 smithi045 ceph-mon[33299]: Deploying daemon mds.j on smithi032 2024-05-03T18:44:05.855 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:05 smithi032 ceph-mon[29763]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:44:05.855 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:05 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:05.855 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:05 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:05.855 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:05 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:05.855 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:05 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T18:44:05.855 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:05 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T18:44:05.855 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:05 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:44:05.856 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:05 smithi032 ceph-mon[29763]: Deploying daemon mds.j on smithi032 2024-05-03T18:44:06.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:05 smithi113 ceph-mon[33540]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:44:06.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:05 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:06.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:05 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:06.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:05 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:06.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:05 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T18:44:06.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:05 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T18:44:06.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:05 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:44:06.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:05 smithi113 ceph-mon[33540]: Deploying daemon mds.j on smithi032 2024-05-03T18:44:06.747 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:06 smithi045 ceph-mon[33299]: mds.? [v2:172.21.15.45:6832/1629869435,v1:172.21.15.45:6833/1629869435] up:boot 2024-05-03T18:44:06.747 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:06 smithi045 ceph-mon[33299]: fsmap 2 up:standby 2024-05-03T18:44:06.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:06 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-03T18:44:06.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:06 smithi045 ceph-mon[33299]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:44:06.842 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:06 smithi032 ceph-mon[29763]: mds.? [v2:172.21.15.45:6832/1629869435,v1:172.21.15.45:6833/1629869435] up:boot 2024-05-03T18:44:06.843 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:06 smithi032 ceph-mon[29763]: fsmap 2 up:standby 2024-05-03T18:44:06.843 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:06 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-03T18:44:06.843 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:06 smithi032 ceph-mon[29763]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:44:07.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:06 smithi113 ceph-mon[33540]: mds.? [v2:172.21.15.45:6832/1629869435,v1:172.21.15.45:6833/1629869435] up:boot 2024-05-03T18:44:07.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:06 smithi113 ceph-mon[33540]: fsmap 2 up:standby 2024-05-03T18:44:07.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:06 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-03T18:44:07.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:06 smithi113 ceph-mon[33540]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:44:07.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:07.175+0000 7f7b36b57700 1 -- 172.21.15.32:0/2734620266 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7b3006c630 msgr2=0x7f7b30110360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:44:07.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:07.175+0000 7f7b36b57700 1 --2- 172.21.15.32:0/2734620266 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7b3006c630 0x7f7b30110360 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f7b2800b0d0 tx=0x7f7b2800b3e0 comp rx=0 tx=0).stop 2024-05-03T18:44:07.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:07.175+0000 7f7b36b57700 1 -- 172.21.15.32:0/2734620266 shutdown_connections 2024-05-03T18:44:07.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:07.175+0000 7f7b36b57700 1 --2- 172.21.15.32:0/2734620266 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7b3006c630 0x7f7b30110360 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:07.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:07.175+0000 7f7b36b57700 1 --2- 172.21.15.32:0/2734620266 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7b30109590 0x7f7b30109970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:07.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:07.175+0000 7f7b36b57700 1 --2- 172.21.15.32:0/2734620266 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7b3006bc70 0x7f7b3006c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:07.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:07.175+0000 7f7b36b57700 1 -- 172.21.15.32:0/2734620266 >> 172.21.15.32:0/2734620266 conn(0x7f7b30079440 msgr2=0x7f7b30079850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:44:07.177 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:07.175+0000 7f7b36b57700 1 -- 172.21.15.32:0/2734620266 shutdown_connections 2024-05-03T18:44:07.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:07.175+0000 7f7b36b57700 1 -- 172.21.15.32:0/2734620266 wait complete. 2024-05-03T18:44:07.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:07.175+0000 7f7b36b57700 1 Processor -- start 2024-05-03T18:44:07.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:07.176+0000 7f7b36b57700 1 -- start start 2024-05-03T18:44:07.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:07.176+0000 7f7b36b57700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7b3006bc70 0x7f7b3019f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:07.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:07.176+0000 7f7b36b57700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7b3006c630 0x7f7b3019fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:07.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:07.176+0000 7f7b36b57700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7b30109590 0x7f7b301999c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:07.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:07.176+0000 7f7b36b57700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f7b30113700 con 0x7f7b30109590 2024-05-03T18:44:07.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:07.176+0000 7f7b36b57700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7b30113580 con 0x7f7b3006bc70 2024-05-03T18:44:07.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:07.176+0000 7f7b36b57700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f7b30113880 con 0x7f7b3006c630 2024-05-03T18:44:07.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:07.176+0000 7f7b350f4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7b30109590 0x7f7b301999c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:44:07.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:07.176+0000 7f7b350f4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7b30109590 0x7f7b301999c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42174/0 (socket says 172.21.15.32:42174) 2024-05-03T18:44:07.178 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:07.176+0000 7f7b350f4700 1 -- 172.21.15.32:0/3518363207 learned_addr learned my addr 172.21.15.32:0/3518363207 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:44:07.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:07.176+0000 7f7b348f3700 1 --2- 172.21.15.32:0/3518363207 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7b3006bc70 0x7f7b3019f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:44:07.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:07.176+0000 7f7b2ffff700 1 --2- 172.21.15.32:0/3518363207 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7b3006c630 0x7f7b3019fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:44:07.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:07.176+0000 7f7b350f4700 1 -- 172.21.15.32:0/3518363207 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7b3006c630 msgr2=0x7f7b3019fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:44:07.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:07.176+0000 7f7b350f4700 1 --2- 172.21.15.32:0/3518363207 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7b3006c630 0x7f7b3019fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:07.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:07.176+0000 7f7b350f4700 1 -- 172.21.15.32:0/3518363207 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7b3006bc70 msgr2=0x7f7b3019f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:44:07.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:07.176+0000 7f7b350f4700 1 --2- 172.21.15.32:0/3518363207 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7b3006bc70 0x7f7b3019f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:07.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:07.176+0000 7f7b350f4700 1 -- 172.21.15.32:0/3518363207 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b3019a120 con 0x7f7b30109590 2024-05-03T18:44:07.179 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:07.176+0000 7f7b350f4700 1 --2- 172.21.15.32:0/3518363207 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7b30109590 0x7f7b301999c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f7b280095c0 tx=0x7f7b28007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:44:07.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:07.177+0000 7f7b2dffb700 1 -- 172.21.15.32:0/3518363207 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b2801f070 con 0x7f7b30109590 2024-05-03T18:44:07.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:07.177+0000 7f7b2dffb700 1 -- 172.21.15.32:0/3518363207 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7b28007e20 con 0x7f7b30109590 2024-05-03T18:44:07.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:07.177+0000 7f7b36b57700 1 -- 172.21.15.32:0/3518363207 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b3019a3b0 con 0x7f7b30109590 2024-05-03T18:44:07.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:07.177+0000 7f7b2dffb700 1 -- 172.21.15.32:0/3518363207 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b2801b4a0 con 0x7f7b30109590 2024-05-03T18:44:07.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:07.177+0000 7f7b36b57700 1 -- 172.21.15.32:0/3518363207 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b301a66d0 con 0x7f7b30109590 2024-05-03T18:44:07.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:07.178+0000 7f7b36b57700 1 -- 172.21.15.32:0/3518363207 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b30104c00 con 0x7f7b30109590 2024-05-03T18:44:07.180 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:07.179+0000 7f7b2dffb700 1 -- 172.21.15.32:0/3518363207 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f7b28022070 con 0x7f7b30109590 2024-05-03T18:44:07.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:07.180+0000 7f7b2dffb700 1 --2- 172.21.15.32:0/3518363207 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f7b180b2880 0x7f7b180b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:07.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:07.180+0000 7f7b348f3700 1 --2- 172.21.15.32:0/3518363207 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f7b180b2880 0x7f7b180b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:44:07.181 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:07.180+0000 7f7b2dffb700 1 -- 172.21.15.32:0/3518363207 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f7b2800f020 con 0x7f7b30109590 2024-05-03T18:44:07.182 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:07.181+0000 7f7b348f3700 1 --2- 172.21.15.32:0/3518363207 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f7b180b2880 0x7f7b180b4d40 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f7b3019ab00 tx=0x7f7b200073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:44:07.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:07.184+0000 7f7b2dffb700 1 -- 172.21.15.32:0/3518363207 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f7b28029020 con 0x7f7b30109590 2024-05-03T18:44:07.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:07.458+0000 7f7b36b57700 1 -- 172.21.15.32:0/3518363207 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7b3019aef0 con 0x7f7b30109590 2024-05-03T18:44:07.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:07.458+0000 7f7b2dffb700 1 -- 172.21.15.32:0/3518363207 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2221 (secure 0 0 0) 0x7f7b280ab560 con 0x7f7b30109590 2024-05-03T18:44:07.460 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-03T18:44:07.460 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24380,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/1629869435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":1629869435},{"type":"v1","addr":"172.21.15.45:6833","nonce":1629869435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6833/3323558791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":3323558791},{"type":"v1","addr":"172.21.15.113:6833","nonce":3323558791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-05-03T18:44:07.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:07.460+0000 7f7b36b57700 1 -- 172.21.15.32:0/3518363207 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f7b180b2880 msgr2=0x7f7b180b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:44:07.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:07.460+0000 7f7b36b57700 1 --2- 172.21.15.32:0/3518363207 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f7b180b2880 0x7f7b180b4d40 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f7b3019ab00 tx=0x7f7b200073d0 comp rx=0 tx=0).stop 2024-05-03T18:44:07.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:07.460+0000 7f7b36b57700 1 -- 172.21.15.32:0/3518363207 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7b30109590 msgr2=0x7f7b301999c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:44:07.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:07.460+0000 7f7b36b57700 1 --2- 172.21.15.32:0/3518363207 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7b30109590 0x7f7b301999c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f7b280095c0 tx=0x7f7b28007a20 comp rx=0 tx=0).stop 2024-05-03T18:44:07.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:07.460+0000 7f7b36b57700 1 -- 172.21.15.32:0/3518363207 shutdown_connections 2024-05-03T18:44:07.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:07.460+0000 7f7b36b57700 1 --2- 172.21.15.32:0/3518363207 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f7b180b2880 0x7f7b180b4d40 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:07.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:07.460+0000 7f7b36b57700 1 --2- 172.21.15.32:0/3518363207 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7b30109590 0x7f7b301999c0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:07.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:07.460+0000 7f7b36b57700 1 --2- 172.21.15.32:0/3518363207 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7b3006bc70 0x7f7b3019f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:07.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:07.460+0000 7f7b36b57700 1 --2- 172.21.15.32:0/3518363207 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7b3006c630 0x7f7b3019fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:07.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:07.460+0000 7f7b36b57700 1 -- 172.21.15.32:0/3518363207 >> 172.21.15.32:0/3518363207 conn(0x7f7b30079440 msgr2=0x7f7b30100f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:44:07.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:07.461+0000 7f7b36b57700 1 -- 172.21.15.32:0/3518363207 shutdown_connections 2024-05-03T18:44:07.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:07.461+0000 7f7b36b57700 1 -- 172.21.15.32:0/3518363207 wait complete. 2024-05-03T18:44:07.463 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 3 2024-05-03T18:44:07.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:07 smithi032 ceph-mon[29763]: from='client.? 172.21.15.32:0/3518363207' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:44:07.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:07 smithi045 ceph-mon[33299]: from='client.? 172.21.15.32:0/3518363207' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:44:08.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:07 smithi113 ceph-mon[33540]: from='client.? 172.21.15.32:0/3518363207' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:44:08.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:08 smithi032 ceph-mon[29763]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:44:08.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:08 smithi045 ceph-mon[33299]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:44:09.049 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:08 smithi113 ceph-mon[33540]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:44:09.568 INFO:journalctl@ceph.mds.j.smithi032.stdout:May 03 18:44:09 smithi032 systemd[1]: Starting Ceph mds.j for f500455e-097b-11ef-bc97-c7b262605968... 2024-05-03T18:44:09.972 INFO:journalctl@ceph.mds.j.smithi032.stdout:May 03 18:44:09 smithi032 podman[67698]: 2024-05-03 18:44:09.612449293 +0000 UTC m=+0.018576624 image pull ee4d846202496ba6096593e8e8becc0700a5eb7ebd95536031187f5f337d4012 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd 2024-05-03T18:44:09.973 INFO:journalctl@ceph.mds.j.smithi032.stdout:May 03 18:44:09 smithi032 podman[67698]: 2024-05-03 18:44:09.709120964 +0000 UTC m=+0.115248284 container create 8bfc26b180308fd5a31275efad80c39c245d6bfe201dd92b5d4a041c864660f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-mds-j, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240430, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-05-03T18:44:09.973 INFO:journalctl@ceph.mds.j.smithi032.stdout:May 03 18:44:09 smithi032 podman[67698]: 2024-05-03 18:44:09.945001528 +0000 UTC m=+0.351128848 container init 8bfc26b180308fd5a31275efad80c39c245d6bfe201dd92b5d4a041c864660f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-mds-j, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240430) 2024-05-03T18:44:09.973 INFO:journalctl@ceph.mds.j.smithi032.stdout:May 03 18:44:09 smithi032 podman[67698]: 2024-05-03 18:44:09.94776807 +0000 UTC m=+0.353895393 container start 8bfc26b180308fd5a31275efad80c39c245d6bfe201dd92b5d4a041c864660f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-mds-j, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240430, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-05-03T18:44:09.973 INFO:journalctl@ceph.mds.j.smithi032.stdout:May 03 18:44:09 smithi032 ceph-mds[67718]: -- 172.21.15.32:0/3368264063 <== mon.0 v2:172.21.15.32:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55983506ba40 con 0x55983503e800 2024-05-03T18:44:10.318 INFO:journalctl@ceph.mds.j.smithi032.stdout:May 03 18:44:09 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-mds-j[67714]: starting mds.j at 2024-05-03T18:44:10.318 INFO:journalctl@ceph.mds.j.smithi032.stdout:May 03 18:44:10 smithi032 bash[67698]: 8bfc26b180308fd5a31275efad80c39c245d6bfe201dd92b5d4a041c864660f7 2024-05-03T18:44:10.318 INFO:journalctl@ceph.mds.j.smithi032.stdout:May 03 18:44:10 smithi032 systemd[1]: Started Ceph mds.j for f500455e-097b-11ef-bc97-c7b262605968. 2024-05-03T18:44:10.600 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid f500455e-097b-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-03T18:44:11.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:11 smithi113 ceph-mon[33540]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:44:11.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:11 smithi113 ceph-mon[33540]: mds.? [v2:172.21.15.32:6834/535599929,v1:172.21.15.32:6835/535599929] up:boot 2024-05-03T18:44:11.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:11 smithi113 ceph-mon[33540]: fsmap 3 up:standby 2024-05-03T18:44:11.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:11 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-03T18:44:11.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:11 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:11.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:11 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:11.302 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:11 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:11.302 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:11 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T18:44:11.302 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:11 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T18:44:11.302 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:11 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:44:11.302 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:11 smithi113 ceph-mon[33540]: Deploying daemon mds.d on smithi032 2024-05-03T18:44:11.303 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:44:11.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:11 smithi032 ceph-mon[29763]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:44:11.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:11 smithi032 ceph-mon[29763]: mds.? [v2:172.21.15.32:6834/535599929,v1:172.21.15.32:6835/535599929] up:boot 2024-05-03T18:44:11.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:11 smithi032 ceph-mon[29763]: fsmap 3 up:standby 2024-05-03T18:44:11.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:11 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-03T18:44:11.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:11 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:11.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:11 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:11.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:11 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:11.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:11 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T18:44:11.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:11 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T18:44:11.319 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:11 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:44:11.319 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:11 smithi032 ceph-mon[29763]: Deploying daemon mds.d on smithi032 2024-05-03T18:44:11.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:11 smithi045 ceph-mon[33299]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:44:11.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:11 smithi045 ceph-mon[33299]: mds.? [v2:172.21.15.32:6834/535599929,v1:172.21.15.32:6835/535599929] up:boot 2024-05-03T18:44:11.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:11 smithi045 ceph-mon[33299]: fsmap 3 up:standby 2024-05-03T18:44:11.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:11 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-03T18:44:11.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:11 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:11.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:11 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:11.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:11 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:11.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:11 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T18:44:11.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:11 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T18:44:11.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:11 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:44:11.499 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:11 smithi045 ceph-mon[33299]: Deploying daemon mds.d on smithi032 2024-05-03T18:44:13.255 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:13.251+0000 7ff6c50a4700 1 -- 172.21.15.32:0/2558156625 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6c0109510 msgr2=0x7ff6c01098f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:44:13.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:13.251+0000 7ff6c50a4700 1 --2- 172.21.15.32:0/2558156625 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6c0109510 0x7ff6c01098f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff6b0009990 tx=0x7ff6b0009ca0 comp rx=0 tx=0).stop 2024-05-03T18:44:13.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:13.251+0000 7ff6c50a4700 1 -- 172.21.15.32:0/2558156625 shutdown_connections 2024-05-03T18:44:13.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:13.251+0000 7ff6c50a4700 1 --2- 172.21.15.32:0/2558156625 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff6c006bb40 0x7ff6c0110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:13.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:13.251+0000 7ff6c50a4700 1 --2- 172.21.15.32:0/2558156625 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6c0109510 0x7ff6c01098f0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:13.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:13.251+0000 7ff6c50a4700 1 --2- 172.21.15.32:0/2558156625 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff6c006b180 0x7ff6c006b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:13.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:13.251+0000 7ff6c50a4700 1 -- 172.21.15.32:0/2558156625 >> 172.21.15.32:0/2558156625 conn(0x7ff6c0079350 msgr2=0x7ff6c0079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:44:13.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:13.251+0000 7ff6c50a4700 1 -- 172.21.15.32:0/2558156625 shutdown_connections 2024-05-03T18:44:13.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:13.251+0000 7ff6c50a4700 1 -- 172.21.15.32:0/2558156625 wait complete. 2024-05-03T18:44:13.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:13.252+0000 7ff6c50a4700 1 Processor -- start 2024-05-03T18:44:13.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:13.252+0000 7ff6c50a4700 1 -- start start 2024-05-03T18:44:13.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:13.252+0000 7ff6c50a4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6c006b180 0x7ff6c0102470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:13.256 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:13.252+0000 7ff6c50a4700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff6c006bb40 0x7ff6c01029b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:13.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:13.252+0000 7ff6c50a4700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff6c0109510 0x7ff6c0102ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:13.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:13.252+0000 7ff6c50a4700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff6c0113710 con 0x7ff6c006b180 2024-05-03T18:44:13.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:13.252+0000 7ff6c50a4700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff6c0113590 con 0x7ff6c0109510 2024-05-03T18:44:13.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:13.252+0000 7ff6c50a4700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7ff6c0113890 con 0x7ff6c006bb40 2024-05-03T18:44:13.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:13.252+0000 7ff6bed9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6c006b180 0x7ff6c0102470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:44:13.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:13.252+0000 7ff6bed9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6c006b180 0x7ff6c0102470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42200/0 (socket says 172.21.15.32:42200) 2024-05-03T18:44:13.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:13.252+0000 7ff6bed9d700 1 -- 172.21.15.32:0/3273632585 learned_addr learned my addr 172.21.15.32:0/3273632585 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:44:13.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:13.252+0000 7ff6be59c700 1 --2- 172.21.15.32:0/3273632585 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff6c006bb40 0x7ff6c01029b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:44:13.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:13.253+0000 7ff6bf59e700 1 --2- 172.21.15.32:0/3273632585 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff6c0109510 0x7ff6c0102ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:44:13.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:13.253+0000 7ff6bed9d700 1 -- 172.21.15.32:0/3273632585 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff6c006bb40 msgr2=0x7ff6c01029b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:44:13.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:13.253+0000 7ff6bed9d700 1 --2- 172.21.15.32:0/3273632585 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff6c006bb40 0x7ff6c01029b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:13.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:13.253+0000 7ff6bed9d700 1 -- 172.21.15.32:0/3273632585 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff6c0109510 msgr2=0x7ff6c0102ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:44:13.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:13.253+0000 7ff6bed9d700 1 --2- 172.21.15.32:0/3273632585 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff6c0109510 0x7ff6c0102ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:13.257 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:13.253+0000 7ff6bed9d700 1 -- 172.21.15.32:0/3273632585 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6c01a8170 con 0x7ff6c006b180 2024-05-03T18:44:13.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:13.253+0000 7ff6bf59e700 1 --2- 172.21.15.32:0/3273632585 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff6c0109510 0x7ff6c0102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:44:13.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:13.253+0000 7ff6bed9d700 1 --2- 172.21.15.32:0/3273632585 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6c006b180 0x7ff6c0102470 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ff6b0005910 tx=0x7ff6b00047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:44:13.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:13.253+0000 7ff6a7fff700 1 -- 172.21.15.32:0/3273632585 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6b0027070 con 0x7ff6c006b180 2024-05-03T18:44:13.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:13.253+0000 7ff6c50a4700 1 -- 172.21.15.32:0/3273632585 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6c01a8400 con 0x7ff6c006b180 2024-05-03T18:44:13.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:13.253+0000 7ff6a7fff700 1 -- 172.21.15.32:0/3273632585 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6b0003f20 con 0x7ff6c006b180 2024-05-03T18:44:13.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:13.253+0000 7ff6a7fff700 1 -- 172.21.15.32:0/3273632585 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6b0022c70 con 0x7ff6c006b180 2024-05-03T18:44:13.258 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:13.254+0000 7ff6c50a4700 1 -- 172.21.15.32:0/3273632585 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6c01a88b0 con 0x7ff6c006b180 2024-05-03T18:44:13.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:13.254+0000 7ff6c50a4700 1 -- 172.21.15.32:0/3273632585 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6c010b9f0 con 0x7ff6c006b180 2024-05-03T18:44:13.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:13.256+0000 7ff6a7fff700 1 -- 172.21.15.32:0/3273632585 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7ff6b002a070 con 0x7ff6c006b180 2024-05-03T18:44:13.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:13.257+0000 7ff6a7fff700 1 --2- 172.21.15.32:0/3273632585 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7ff6a80b2880 0x7ff6a80b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:13.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:13.257+0000 7ff6be59c700 1 --2- 172.21.15.32:0/3273632585 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7ff6a80b2880 0x7ff6a80b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:44:13.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:13.257+0000 7ff6a7fff700 1 -- 172.21.15.32:0/3273632585 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7ff6b00e2c80 con 0x7ff6c006b180 2024-05-03T18:44:13.259 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:13.257+0000 7ff6be59c700 1 --2- 172.21.15.32:0/3273632585 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7ff6a80b2880 0x7ff6a80b4d40 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ff6c0079a30 tx=0x7ff6b400b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:44:13.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:13.261+0000 7ff6a7fff700 1 -- 172.21.15.32:0/3273632585 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7ff6b0013070 con 0x7ff6c006b180 2024-05-03T18:44:13.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:13 smithi113 ceph-mon[33540]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:44:13.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:13 smithi032 ceph-mon[29763]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:44:13.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:13 smithi045 ceph-mon[33299]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:44:13.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:13.544+0000 7ff6c50a4700 1 -- 172.21.15.32:0/3273632585 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff6c01039a0 con 0x7ff6c006b180 2024-05-03T18:44:13.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:13.545+0000 7ff6a7fff700 1 -- 172.21.15.32:0/3273632585 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3043 (secure 0 0 0) 0x7ff6b00aae30 con 0x7ff6c006b180 2024-05-03T18:44:13.547 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-03T18:44:13.547 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/535599929","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":535599929},{"type":"v1","addr":"172.21.15.32:6835","nonce":535599929}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24380,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/1629869435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":1629869435},{"type":"v1","addr":"172.21.15.45:6833","nonce":1629869435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6833/3323558791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":3323558791},{"type":"v1","addr":"172.21.15.113:6833","nonce":3323558791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-05-03T18:44:13.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:13.547+0000 7ff6c50a4700 1 -- 172.21.15.32:0/3273632585 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7ff6a80b2880 msgr2=0x7ff6a80b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:44:13.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:13.547+0000 7ff6c50a4700 1 --2- 172.21.15.32:0/3273632585 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7ff6a80b2880 0x7ff6a80b4d40 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ff6c0079a30 tx=0x7ff6b400b3c0 comp rx=0 tx=0).stop 2024-05-03T18:44:13.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:13.547+0000 7ff6c50a4700 1 -- 172.21.15.32:0/3273632585 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6c006b180 msgr2=0x7ff6c0102470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:44:13.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:13.547+0000 7ff6c50a4700 1 --2- 172.21.15.32:0/3273632585 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6c006b180 0x7ff6c0102470 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ff6b0005910 tx=0x7ff6b00047b0 comp rx=0 tx=0).stop 2024-05-03T18:44:13.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:13.547+0000 7ff6c50a4700 1 -- 172.21.15.32:0/3273632585 shutdown_connections 2024-05-03T18:44:13.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:13.547+0000 7ff6c50a4700 1 --2- 172.21.15.32:0/3273632585 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7ff6a80b2880 0x7ff6a80b4d40 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:13.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:13.547+0000 7ff6c50a4700 1 --2- 172.21.15.32:0/3273632585 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff6c0109510 0x7ff6c0102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:13.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:13.547+0000 7ff6c50a4700 1 --2- 172.21.15.32:0/3273632585 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6c006b180 0x7ff6c0102470 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:13.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:13.547+0000 7ff6c50a4700 1 --2- 172.21.15.32:0/3273632585 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff6c006bb40 0x7ff6c01029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:13.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:13.547+0000 7ff6c50a4700 1 -- 172.21.15.32:0/3273632585 >> 172.21.15.32:0/3273632585 conn(0x7ff6c0079350 msgr2=0x7ff6c01011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:44:13.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:13.547+0000 7ff6c50a4700 1 -- 172.21.15.32:0/3273632585 shutdown_connections 2024-05-03T18:44:13.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:13.547+0000 7ff6c50a4700 1 -- 172.21.15.32:0/3273632585 wait complete. 2024-05-03T18:44:13.550 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 4 2024-05-03T18:44:14.253 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:14 smithi032 ceph-mon[29763]: from='client.? 172.21.15.32:0/3273632585' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:44:14.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:14 smithi113 ceph-mon[33540]: from='client.? 172.21.15.32:0/3273632585' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:44:14.497 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:14 smithi045 ceph-mon[33299]: from='client.? 172.21.15.32:0/3273632585' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:44:14.568 INFO:journalctl@ceph.mds.d.smithi032.stdout:May 03 18:44:14 smithi032 systemd[1]: Started Ceph mds.d for f500455e-097b-11ef-bc97-c7b262605968. 2024-05-03T18:44:15.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:15 smithi045 ceph-mon[33299]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:44:15.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:15 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:15.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:15 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:15.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:15 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:15.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:15 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T18:44:15.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:15 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T18:44:15.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:15 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:44:15.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:15 smithi045 ceph-mon[33299]: Deploying daemon mds.e on smithi045 2024-05-03T18:44:15.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:15 smithi045 ceph-mon[33299]: mds.? [v2:172.21.15.32:6836/3101679868,v1:172.21.15.32:6837/3101679868] up:boot 2024-05-03T18:44:15.749 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:15 smithi045 ceph-mon[33299]: fsmap 4 up:standby 2024-05-03T18:44:15.749 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:15 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-03T18:44:15.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:15 smithi113 ceph-mon[33540]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:44:15.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:15 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:15.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:15 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:15.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:15 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:15.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:15 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T18:44:15.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:15 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T18:44:15.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:15 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:44:15.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:15 smithi113 ceph-mon[33540]: Deploying daemon mds.e on smithi045 2024-05-03T18:44:15.802 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:15 smithi113 ceph-mon[33540]: mds.? [v2:172.21.15.32:6836/3101679868,v1:172.21.15.32:6837/3101679868] up:boot 2024-05-03T18:44:15.802 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:15 smithi113 ceph-mon[33540]: fsmap 4 up:standby 2024-05-03T18:44:15.802 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:15 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-03T18:44:15.819 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:15 smithi032 ceph-mon[29763]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:44:15.819 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:15 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:15.819 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:15 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:15.819 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:15 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:15.819 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:15 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T18:44:15.819 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:15 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T18:44:15.820 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:15 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:44:15.820 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:15 smithi032 ceph-mon[29763]: Deploying daemon mds.e on smithi045 2024-05-03T18:44:15.820 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:15 smithi032 ceph-mon[29763]: mds.? [v2:172.21.15.32:6836/3101679868,v1:172.21.15.32:6837/3101679868] up:boot 2024-05-03T18:44:15.820 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:15 smithi032 ceph-mon[29763]: fsmap 4 up:standby 2024-05-03T18:44:15.820 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:15 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-03T18:44:16.168 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid f500455e-097b-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-03T18:44:16.435 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:44:16.498 INFO:journalctl@ceph.mds.e.smithi045.stdout:May 03 18:44:16 smithi045 systemd[1]: Starting Ceph mds.e for f500455e-097b-11ef-bc97-c7b262605968... 2024-05-03T18:44:16.878 INFO:journalctl@ceph.mds.e.smithi045.stdout:May 03 18:44:16 smithi045 podman[63152]: 2024-05-03 18:44:16.583964122 +0000 UTC m=+0.110986971 container create 5eec80064e6eb87f440e59a997bb707af8b0e384603302a32628d96f766a582d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-mds-e, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240430, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-05-03T18:44:16.878 INFO:journalctl@ceph.mds.e.smithi045.stdout:May 03 18:44:16 smithi045 podman[63152]: 2024-05-03 18:44:16.491150807 +0000 UTC m=+0.018173676 image pull ee4d846202496ba6096593e8e8becc0700a5eb7ebd95536031187f5f337d4012 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd 2024-05-03T18:44:16.878 INFO:journalctl@ceph.mds.e.smithi045.stdout:May 03 18:44:16 smithi045 podman[63152]: 2024-05-03 18:44:16.802800183 +0000 UTC m=+0.329823034 container init 5eec80064e6eb87f440e59a997bb707af8b0e384603302a32628d96f766a582d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-mds-e, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240430, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True) 2024-05-03T18:44:16.878 INFO:journalctl@ceph.mds.e.smithi045.stdout:May 03 18:44:16 smithi045 podman[63152]: 2024-05-03 18:44:16.805597358 +0000 UTC m=+0.332620208 container start 5eec80064e6eb87f440e59a997bb707af8b0e384603302a32628d96f766a582d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-mds-e, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240430, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-03T18:44:16.878 INFO:journalctl@ceph.mds.e.smithi045.stdout:May 03 18:44:16 smithi045 ceph-mds[63171]: -- 172.21.15.45:0/2012184997 <== mon.2 v2:172.21.15.45:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562932753a40 con 0x562932726c00 2024-05-03T18:44:16.879 INFO:journalctl@ceph.mds.e.smithi045.stdout:May 03 18:44:16 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-mds-e[63167]: starting mds.e at 2024-05-03T18:44:16.879 INFO:journalctl@ceph.mds.e.smithi045.stdout:May 03 18:44:16 smithi045 bash[63152]: 5eec80064e6eb87f440e59a997bb707af8b0e384603302a32628d96f766a582d 2024-05-03T18:44:17.203 INFO:journalctl@ceph.mds.e.smithi045.stdout:May 03 18:44:16 smithi045 systemd[1]: Started Ceph mds.e for f500455e-097b-11ef-bc97-c7b262605968. 2024-05-03T18:44:17.497 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:17 smithi045 ceph-mon[33299]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:44:17.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:17 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:17.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:17 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:17.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:17 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:17.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:17 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T18:44:17.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:17 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T18:44:17.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:17 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:44:17.715 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:17 smithi032 ceph-mon[29763]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:44:17.716 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:17 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:17.716 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:17 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:17.716 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:17 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:17.716 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:17 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T18:44:17.716 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:17 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T18:44:17.716 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:17 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:44:17.720 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:17 smithi113 ceph-mon[33540]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:44:17.721 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:17 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:17.721 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:17 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:17.721 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:17 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:17.721 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:17 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T18:44:17.721 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:17 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T18:44:17.721 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:17 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:44:17.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:17.933+0000 7fd8a5e41700 1 -- 172.21.15.32:0/2924033087 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8a0104100 msgr2=0x7fd8a0108120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:44:17.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:17.933+0000 7fd8a5e41700 1 --2- 172.21.15.32:0/2924033087 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8a0104100 0x7fd8a0108120 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fd8940099c0 tx=0x7fd894009cd0 comp rx=0 tx=0).stop 2024-05-03T18:44:17.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:17.933+0000 7fd8a5e41700 1 -- 172.21.15.32:0/2924033087 shutdown_connections 2024-05-03T18:44:17.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:17.933+0000 7fd8a5e41700 1 --2- 172.21.15.32:0/2924033087 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd8a0108850 0x7fd8a01103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:17.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:17.933+0000 7fd8a5e41700 1 --2- 172.21.15.32:0/2924033087 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd8a0103750 0x7fd8a0103b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:17.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:17.933+0000 7fd8a5e41700 1 --2- 172.21.15.32:0/2924033087 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8a0104100 0x7fd8a0108120 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:17.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:17.933+0000 7fd8a5e41700 1 -- 172.21.15.32:0/2924033087 >> 172.21.15.32:0/2924033087 conn(0x7fd8a007a600 msgr2=0x7fd8a007aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:44:17.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:17.933+0000 7fd8a5e41700 1 -- 172.21.15.32:0/2924033087 shutdown_connections 2024-05-03T18:44:17.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:17.933+0000 7fd8a5e41700 1 -- 172.21.15.32:0/2924033087 wait complete. 2024-05-03T18:44:17.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:17.933+0000 7fd8a5e41700 1 Processor -- start 2024-05-03T18:44:17.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:17.934+0000 7fd8a5e41700 1 -- start start 2024-05-03T18:44:17.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:17.934+0000 7fd8a5e41700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd8a0103750 0x7fd8a01a19b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:17.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:17.934+0000 7fd8a5e41700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd8a0104100 0x7fd8a01a1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:17.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:17.934+0000 7fd8a5e41700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8a0108850 0x7fd8a019bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:17.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:17.934+0000 7fd8a5e41700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd8a0076120 con 0x7fd8a0108850 2024-05-03T18:44:17.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:17.934+0000 7fd8a5e41700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd8a0075fa0 con 0x7fd8a0103750 2024-05-03T18:44:17.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:17.934+0000 7fd8a5e41700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fd8a00762a0 con 0x7fd8a0104100 2024-05-03T18:44:17.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:17.934+0000 7fd89ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8a0108850 0x7fd8a019bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:44:17.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:17.934+0000 7fd89ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8a0108850 0x7fd8a019bd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38176/0 (socket says 172.21.15.32:38176) 2024-05-03T18:44:17.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:17.934+0000 7fd89ffff700 1 -- 172.21.15.32:0/1169856014 learned_addr learned my addr 172.21.15.32:0/1169856014 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:44:17.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:17.934+0000 7fd89f7fe700 1 --2- 172.21.15.32:0/1169856014 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd8a0103750 0x7fd8a01a19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:44:17.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:17.934+0000 7fd89effd700 1 --2- 172.21.15.32:0/1169856014 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd8a0104100 0x7fd8a01a1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:44:17.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:17.934+0000 7fd89ffff700 1 -- 172.21.15.32:0/1169856014 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd8a0104100 msgr2=0x7fd8a01a1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:44:17.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:17.934+0000 7fd89ffff700 1 --2- 172.21.15.32:0/1169856014 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd8a0104100 0x7fd8a01a1ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:17.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:17.934+0000 7fd89ffff700 1 -- 172.21.15.32:0/1169856014 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd8a0103750 msgr2=0x7fd8a01a19b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:44:17.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:17.934+0000 7fd89ffff700 1 --2- 172.21.15.32:0/1169856014 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd8a0103750 0x7fd8a01a19b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:17.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:17.934+0000 7fd89ffff700 1 -- 172.21.15.32:0/1169856014 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8a019c4d0 con 0x7fd8a0108850 2024-05-03T18:44:17.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:17.935+0000 7fd89ffff700 1 --2- 172.21.15.32:0/1169856014 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8a0108850 0x7fd8a019bd00 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fd89800ef30 tx=0x7fd89800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:44:17.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:17.935+0000 7fd89cff9700 1 -- 172.21.15.32:0/1169856014 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8980092e0 con 0x7fd8a0108850 2024-05-03T18:44:17.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:17.935+0000 7fd89cff9700 1 -- 172.21.15.32:0/1169856014 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd898004bb0 con 0x7fd8a0108850 2024-05-03T18:44:17.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:17.935+0000 7fd8a5e41700 1 -- 172.21.15.32:0/1169856014 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8a019c760 con 0x7fd8a0108850 2024-05-03T18:44:17.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:17.935+0000 7fd89cff9700 1 -- 172.21.15.32:0/1169856014 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd898004020 con 0x7fd8a0108850 2024-05-03T18:44:17.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:17.935+0000 7fd8a5e41700 1 -- 172.21.15.32:0/1169856014 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8a01a8810 con 0x7fd8a0108850 2024-05-03T18:44:17.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:17.936+0000 7fd8a5e41700 1 -- 172.21.15.32:0/1169856014 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8a006a190 con 0x7fd8a0108850 2024-05-03T18:44:17.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:17.937+0000 7fd89cff9700 1 -- 172.21.15.32:0/1169856014 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fd89801b030 con 0x7fd8a0108850 2024-05-03T18:44:17.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:17.938+0000 7fd89cff9700 1 --2- 172.21.15.32:0/1169856014 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fd8880b2900 0x7fd8880b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:17.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:17.938+0000 7fd89f7fe700 1 --2- 172.21.15.32:0/1169856014 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fd8880b2900 0x7fd8880b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:44:17.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:17.939+0000 7fd89cff9700 1 -- 172.21.15.32:0/1169856014 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7fd8980e21d0 con 0x7fd8a0108850 2024-05-03T18:44:17.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:17.939+0000 7fd89f7fe700 1 --2- 172.21.15.32:0/1169856014 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fd8880b2900 0x7fd8880b4dc0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd890005950 tx=0x7fd89000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:44:17.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:17.942+0000 7fd89cff9700 1 -- 172.21.15.32:0/1169856014 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fd898022020 con 0x7fd8a0108850 2024-05-03T18:44:18.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:18.210+0000 7fd8a5e41700 1 -- 172.21.15.32:0/1169856014 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd8a01a8ad0 con 0x7fd8a0108850 2024-05-03T18:44:18.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:18.210+0000 7fd89cff9700 1 -- 172.21.15.32:0/1169856014 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4693 (secure 0 0 0) 0x7fd898010040 con 0x7fd8a0108850 2024-05-03T18:44:18.211 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-03T18:44:18.211 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/535599929","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":535599929},{"type":"v1","addr":"172.21.15.32:6835","nonce":535599929}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/3101679868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":3101679868},{"type":"v1","addr":"172.21.15.32:6837","nonce":3101679868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24380,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/1629869435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":1629869435},{"type":"v1","addr":"172.21.15.45:6833","nonce":1629869435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24398,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2997291166","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2997291166},{"type":"v1","addr":"172.21.15.45:6835","nonce":2997291166}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6833/3323558791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":3323558791},{"type":"v1","addr":"172.21.15.113:6833","nonce":3323558791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-05-03T18:44:18.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:18.212+0000 7fd8a5e41700 1 -- 172.21.15.32:0/1169856014 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fd8880b2900 msgr2=0x7fd8880b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:44:18.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:18.213+0000 7fd8a5e41700 1 --2- 172.21.15.32:0/1169856014 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fd8880b2900 0x7fd8880b4dc0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd890005950 tx=0x7fd89000b410 comp rx=0 tx=0).stop 2024-05-03T18:44:18.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:18.213+0000 7fd8a5e41700 1 -- 172.21.15.32:0/1169856014 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8a0108850 msgr2=0x7fd8a019bd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:44:18.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:18.213+0000 7fd8a5e41700 1 --2- 172.21.15.32:0/1169856014 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8a0108850 0x7fd8a019bd00 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fd89800ef30 tx=0x7fd89800c3a0 comp rx=0 tx=0).stop 2024-05-03T18:44:18.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:18.213+0000 7fd8a5e41700 1 -- 172.21.15.32:0/1169856014 shutdown_connections 2024-05-03T18:44:18.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:18.213+0000 7fd8a5e41700 1 --2- 172.21.15.32:0/1169856014 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fd8880b2900 0x7fd8880b4dc0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:18.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:18.213+0000 7fd8a5e41700 1 --2- 172.21.15.32:0/1169856014 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8a0108850 0x7fd8a019bd00 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:18.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:18.213+0000 7fd8a5e41700 1 --2- 172.21.15.32:0/1169856014 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd8a0103750 0x7fd8a01a19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:18.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:18.213+0000 7fd8a5e41700 1 --2- 172.21.15.32:0/1169856014 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd8a0104100 0x7fd8a01a1ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:18.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:18.213+0000 7fd8a5e41700 1 -- 172.21.15.32:0/1169856014 >> 172.21.15.32:0/1169856014 conn(0x7fd8a007a600 msgr2=0x7fd8a0107070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:44:18.214 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:18.213+0000 7fd8a5e41700 1 -- 172.21.15.32:0/1169856014 shutdown_connections 2024-05-03T18:44:18.215 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:18.213+0000 7fd8a5e41700 1 -- 172.21.15.32:0/1169856014 wait complete. 2024-05-03T18:44:18.215 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 6 2024-05-03T18:44:18.559 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:18 smithi032 ceph-mon[29763]: Deploying daemon mds.f on smithi113 2024-05-03T18:44:18.559 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:18 smithi032 ceph-mon[29763]: mds.? [v2:172.21.15.45:6834/2997291166,v1:172.21.15.45:6835/2997291166] up:boot 2024-05-03T18:44:18.559 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:18 smithi032 ceph-mon[29763]: fsmap 5 up:standby 2024-05-03T18:44:18.559 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:18 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-03T18:44:18.559 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:18 smithi032 ceph-mon[29763]: from='client.? 172.21.15.32:0/1169856014' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:44:18.673 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:18 smithi113 ceph-mon[33540]: Deploying daemon mds.f on smithi113 2024-05-03T18:44:18.673 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:18 smithi113 ceph-mon[33540]: mds.? [v2:172.21.15.45:6834/2997291166,v1:172.21.15.45:6835/2997291166] up:boot 2024-05-03T18:44:18.673 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:18 smithi113 ceph-mon[33540]: fsmap 5 up:standby 2024-05-03T18:44:18.673 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:18 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-03T18:44:18.673 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:18 smithi113 ceph-mon[33540]: from='client.? 172.21.15.32:0/1169856014' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:44:18.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:18 smithi045 ceph-mon[33299]: Deploying daemon mds.f on smithi113 2024-05-03T18:44:18.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:18 smithi045 ceph-mon[33299]: mds.? [v2:172.21.15.45:6834/2997291166,v1:172.21.15.45:6835/2997291166] up:boot 2024-05-03T18:44:18.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:18 smithi045 ceph-mon[33299]: fsmap 5 up:standby 2024-05-03T18:44:18.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:18 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-03T18:44:18.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:18 smithi045 ceph-mon[33299]: from='client.? 172.21.15.32:0/1169856014' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:44:19.301 INFO:journalctl@ceph.mds.f.smithi113.stdout:May 03 18:44:19 smithi113 systemd[1]: Starting Ceph mds.f for f500455e-097b-11ef-bc97-c7b262605968... 2024-05-03T18:44:19.638 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:19 smithi113 ceph-mon[33540]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:44:19.638 INFO:journalctl@ceph.mds.f.smithi113.stdout:May 03 18:44:19 smithi113 podman[64584]: 2024-05-03 18:44:19.265080393 +0000 UTC m=+0.017908331 image pull ee4d846202496ba6096593e8e8becc0700a5eb7ebd95536031187f5f337d4012 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd 2024-05-03T18:44:19.638 INFO:journalctl@ceph.mds.f.smithi113.stdout:May 03 18:44:19 smithi113 podman[64584]: 2024-05-03 18:44:19.366541698 +0000 UTC m=+0.119369624 container create 33c4af011e7d40c415be21e9fb808310acc03c4034b42cd17f0378ebcd36518d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-mds-f, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240430, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-05-03T18:44:19.638 INFO:journalctl@ceph.mds.f.smithi113.stdout:May 03 18:44:19 smithi113 podman[64584]: 2024-05-03 18:44:19.537660523 +0000 UTC m=+0.290488448 container init 33c4af011e7d40c415be21e9fb808310acc03c4034b42cd17f0378ebcd36518d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-mds-f, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240430, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-05-03T18:44:19.639 INFO:journalctl@ceph.mds.f.smithi113.stdout:May 03 18:44:19 smithi113 podman[64584]: 2024-05-03 18:44:19.540288359 +0000 UTC m=+0.293116292 container start 33c4af011e7d40c415be21e9fb808310acc03c4034b42cd17f0378ebcd36518d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-mds-f, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240430, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-05-03T18:44:19.639 INFO:journalctl@ceph.mds.f.smithi113.stdout:May 03 18:44:19 smithi113 ceph-mds[64603]: -- 172.21.15.113:0/3261560726 <== mon.1 v2:172.21.15.113:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55fd05197a40 con 0x55fd0516ac00 2024-05-03T18:44:19.639 INFO:journalctl@ceph.mds.f.smithi113.stdout:May 03 18:44:19 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-mds-f[64599]: starting mds.f at 2024-05-03T18:44:19.639 INFO:journalctl@ceph.mds.f.smithi113.stdout:May 03 18:44:19 smithi113 bash[64584]: 33c4af011e7d40c415be21e9fb808310acc03c4034b42cd17f0378ebcd36518d 2024-05-03T18:44:19.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:19 smithi045 ceph-mon[33299]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:44:19.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:19 smithi032 ceph-mon[29763]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:44:20.051 INFO:journalctl@ceph.mds.f.smithi113.stdout:May 03 18:44:19 smithi113 systemd[1]: Started Ceph mds.f for f500455e-097b-11ef-bc97-c7b262605968. 2024-05-03T18:44:20.877 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid f500455e-097b-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-03T18:44:20.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:20 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:20.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:20 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:20.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:20 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:20.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:20 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T18:44:20.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:20 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T18:44:20.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:20 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:44:20.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:20 smithi045 ceph-mon[33299]: Deploying daemon mds.c on smithi113 2024-05-03T18:44:20.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:20 smithi045 ceph-mon[33299]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:44:20.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:20 smithi045 ceph-mon[33299]: mds.? [v2:172.21.15.113:6834/2588738276,v1:172.21.15.113:6835/2588738276] up:boot 2024-05-03T18:44:20.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:20 smithi045 ceph-mon[33299]: fsmap 6 up:standby 2024-05-03T18:44:20.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:20 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-03T18:44:21.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:20 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:21.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:20 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:21.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:20 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:21.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:20 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T18:44:21.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:20 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T18:44:21.052 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:20 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:44:21.052 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:20 smithi113 ceph-mon[33540]: Deploying daemon mds.c on smithi113 2024-05-03T18:44:21.052 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:20 smithi113 ceph-mon[33540]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:44:21.052 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:20 smithi113 ceph-mon[33540]: mds.? [v2:172.21.15.113:6834/2588738276,v1:172.21.15.113:6835/2588738276] up:boot 2024-05-03T18:44:21.052 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:20 smithi113 ceph-mon[33540]: fsmap 6 up:standby 2024-05-03T18:44:21.052 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:20 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-03T18:44:21.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:20 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:21.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:20 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:21.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:20 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:21.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:20 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T18:44:21.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:20 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T18:44:21.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:20 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:44:21.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:20 smithi032 ceph-mon[29763]: Deploying daemon mds.c on smithi113 2024-05-03T18:44:21.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:20 smithi032 ceph-mon[29763]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:44:21.069 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:20 smithi032 ceph-mon[29763]: mds.? [v2:172.21.15.113:6834/2588738276,v1:172.21.15.113:6835/2588738276] up:boot 2024-05-03T18:44:21.069 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:20 smithi032 ceph-mon[29763]: fsmap 6 up:standby 2024-05-03T18:44:21.069 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:20 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-03T18:44:21.136 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:44:22.009 INFO:journalctl@ceph.mds.c.smithi113.stdout:May 03 18:44:21 smithi113 systemd[1]: Starting Ceph mds.c for f500455e-097b-11ef-bc97-c7b262605968... 2024-05-03T18:44:22.283 INFO:journalctl@ceph.mds.c.smithi113.stdout:May 03 18:44:22 smithi113 podman[64892]: 2024-05-03 18:44:21.912303107 +0000 UTC m=+0.018532530 image pull ee4d846202496ba6096593e8e8becc0700a5eb7ebd95536031187f5f337d4012 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd 2024-05-03T18:44:22.283 INFO:journalctl@ceph.mds.c.smithi113.stdout:May 03 18:44:22 smithi113 podman[64892]: 2024-05-03 18:44:22.021405101 +0000 UTC m=+0.127634527 container create f3eae32eb8de23b8540fe386f924fb8af776c27337754ec1da616ac117a51584 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-mds-c, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.build-date=20240430, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-05-03T18:44:22.283 INFO:journalctl@ceph.mds.c.smithi113.stdout:May 03 18:44:22 smithi113 podman[64892]: 2024-05-03 18:44:22.282322209 +0000 UTC m=+0.388551626 container init f3eae32eb8de23b8540fe386f924fb8af776c27337754ec1da616ac117a51584 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-mds-c, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240430, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-05-03T18:44:22.551 INFO:journalctl@ceph.mds.c.smithi113.stdout:May 03 18:44:22 smithi113 podman[64892]: 2024-05-03 18:44:22.285003653 +0000 UTC m=+0.391233078 container start f3eae32eb8de23b8540fe386f924fb8af776c27337754ec1da616ac117a51584 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-mds-c, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240430, GIT_CLEAN=True) 2024-05-03T18:44:22.551 INFO:journalctl@ceph.mds.c.smithi113.stdout:May 03 18:44:22 smithi113 ceph-mds[64911]: -- 172.21.15.113:0/1105937674 <== mon.1 v2:172.21.15.113:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562976215a40 con 0x5629761e8c00 2024-05-03T18:44:22.551 INFO:journalctl@ceph.mds.c.smithi113.stdout:May 03 18:44:22 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-mds-c[64907]: starting mds.c at 2024-05-03T18:44:22.551 INFO:journalctl@ceph.mds.c.smithi113.stdout:May 03 18:44:22 smithi113 bash[64892]: f3eae32eb8de23b8540fe386f924fb8af776c27337754ec1da616ac117a51584 2024-05-03T18:44:22.551 INFO:journalctl@ceph.mds.c.smithi113.stdout:May 03 18:44:22 smithi113 systemd[1]: Started Ceph mds.c for f500455e-097b-11ef-bc97-c7b262605968. 2024-05-03T18:44:22.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:22.924+0000 7f41163af700 1 -- 172.21.15.32:0/2745083974 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4110102fd0 msgr2=0x7f4110107020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:44:22.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:22.924+0000 7f41163af700 1 --2- 172.21.15.32:0/2745083974 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4110102fd0 0x7f4110107020 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f40f80099c0 tx=0x7f40f8009cd0 comp rx=0 tx=0).stop 2024-05-03T18:44:22.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:22.924+0000 7f41163af700 1 -- 172.21.15.32:0/2745083974 shutdown_connections 2024-05-03T18:44:22.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:22.924+0000 7f41163af700 1 --2- 172.21.15.32:0/2745083974 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4110107700 0x7f411010f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:22.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:22.924+0000 7f41163af700 1 --2- 172.21.15.32:0/2745083974 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4110102620 0x7f4110102a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:22.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:22.924+0000 7f41163af700 1 --2- 172.21.15.32:0/2745083974 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4110102fd0 0x7f4110107020 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:22.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:22.924+0000 7f41163af700 1 -- 172.21.15.32:0/2745083974 >> 172.21.15.32:0/2745083974 conn(0x7f4110079440 msgr2=0x7f4110079850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:44:22.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:22.924+0000 7f41163af700 1 -- 172.21.15.32:0/2745083974 shutdown_connections 2024-05-03T18:44:22.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:22.924+0000 7f41163af700 1 -- 172.21.15.32:0/2745083974 wait complete. 2024-05-03T18:44:22.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:22.925+0000 7f41163af700 1 Processor -- start 2024-05-03T18:44:22.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:22.925+0000 7f41163af700 1 -- start start 2024-05-03T18:44:22.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:22.925+0000 7f41163af700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4110102620 0x7f411019e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:22.928 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:22.925+0000 7f41163af700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4110102fd0 0x7f411019ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:22.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:22.925+0000 7f41163af700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4110107700 0x7f4110198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:22.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:22.925+0000 7f41163af700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f4110112570 con 0x7f4110107700 2024-05-03T18:44:22.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:22.925+0000 7f41163af700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f41101123f0 con 0x7f4110102620 2024-05-03T18:44:22.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:22.925+0000 7f41163af700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f41101126f0 con 0x7f4110102fd0 2024-05-03T18:44:22.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:22.926+0000 7f411494c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4110107700 0x7f4110198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:44:22.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:22.926+0000 7f411494c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4110107700 0x7f4110198870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38200/0 (socket says 172.21.15.32:38200) 2024-05-03T18:44:22.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:22.926+0000 7f411494c700 1 -- 172.21.15.32:0/2920312011 learned_addr learned my addr 172.21.15.32:0/2920312011 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:44:22.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:22.926+0000 7f410f7fe700 1 --2- 172.21.15.32:0/2920312011 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4110102fd0 0x7f411019ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:44:22.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:22.926+0000 7f410ffff700 1 --2- 172.21.15.32:0/2920312011 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4110102620 0x7f411019e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:44:22.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:22.926+0000 7f411494c700 1 -- 172.21.15.32:0/2920312011 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4110102fd0 msgr2=0x7f411019ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:44:22.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:22.926+0000 7f411494c700 1 --2- 172.21.15.32:0/2920312011 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4110102fd0 0x7f411019ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:22.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:22.926+0000 7f411494c700 1 -- 172.21.15.32:0/2920312011 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4110102620 msgr2=0x7f411019e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:44:22.929 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:22.926+0000 7f411494c700 1 --2- 172.21.15.32:0/2920312011 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4110102620 0x7f411019e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:22.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:22.926+0000 7f411494c700 1 -- 172.21.15.32:0/2920312011 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4110199050 con 0x7f4110107700 2024-05-03T18:44:22.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:22.926+0000 7f411494c700 1 --2- 172.21.15.32:0/2920312011 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4110107700 0x7f4110198870 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f410800ef30 tx=0x7f410800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:44:22.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:22.926+0000 7f410d7fa700 1 -- 172.21.15.32:0/2920312011 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41080092e0 con 0x7f4110107700 2024-05-03T18:44:22.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:22.926+0000 7f410d7fa700 1 -- 172.21.15.32:0/2920312011 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4108004bb0 con 0x7f4110107700 2024-05-03T18:44:22.930 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:22.926+0000 7f41163af700 1 -- 172.21.15.32:0/2920312011 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41101992e0 con 0x7f4110107700 2024-05-03T18:44:22.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:22.926+0000 7f410d7fa700 1 -- 172.21.15.32:0/2920312011 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4108004020 con 0x7f4110107700 2024-05-03T18:44:22.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:22.926+0000 7f41163af700 1 -- 172.21.15.32:0/2920312011 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f411006bbd0 con 0x7f4110107700 2024-05-03T18:44:22.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:22.927+0000 7f41163af700 1 -- 172.21.15.32:0/2920312011 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4110103b00 con 0x7f4110107700 2024-05-03T18:44:22.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:22.929+0000 7f410d7fa700 1 -- 172.21.15.32:0/2920312011 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f410801b030 con 0x7f4110107700 2024-05-03T18:44:22.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:22.929+0000 7f410d7fa700 1 --2- 172.21.15.32:0/2920312011 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f40fc0b2900 0x7f40fc0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:22.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:22.929+0000 7f410ffff700 1 --2- 172.21.15.32:0/2920312011 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f40fc0b2900 0x7f40fc0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:44:22.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:22.930+0000 7f410d7fa700 1 -- 172.21.15.32:0/2920312011 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f41080e2150 con 0x7f4110107700 2024-05-03T18:44:22.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:22.930+0000 7f410ffff700 1 --2- 172.21.15.32:0/2920312011 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f40fc0b2900 0x7f40fc0b4dc0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f410000ac50 tx=0x7f410000a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:44:22.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:22.933+0000 7f410d7fa700 1 -- 172.21.15.32:0/2920312011 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f4108022020 con 0x7f4110107700 2024-05-03T18:44:23.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:23.192+0000 7f41163af700 1 -- 172.21.15.32:0/2920312011 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f411006c600 con 0x7f4110107700 2024-05-03T18:44:23.194 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:23.193+0000 7f410d7fa700 1 -- 172.21.15.32:0/2920312011 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6349 (secure 0 0 0) 0x7f4108010040 con 0x7f4110107700 2024-05-03T18:44:23.194 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-03T18:44:23.194 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/535599929","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":535599929},{"type":"v1","addr":"172.21.15.32:6835","nonce":535599929}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/3101679868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":3101679868},{"type":"v1","addr":"172.21.15.32:6837","nonce":3101679868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24380,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/1629869435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":1629869435},{"type":"v1","addr":"172.21.15.45:6833","nonce":1629869435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24398,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2997291166","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2997291166},{"type":"v1","addr":"172.21.15.45:6835","nonce":2997291166}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6833/3323558791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":3323558791},{"type":"v1","addr":"172.21.15.113:6833","nonce":3323558791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24463,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/2588738276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2588738276},{"type":"v1","addr":"172.21.15.113:6835","nonce":2588738276}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/3677284460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":3677284460},{"type":"v1","addr":"172.21.15.113:6837","nonce":3677284460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-05-03T18:44:23.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:23.195+0000 7f41163af700 1 -- 172.21.15.32:0/2920312011 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f40fc0b2900 msgr2=0x7f40fc0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:44:23.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:23.195+0000 7f41163af700 1 --2- 172.21.15.32:0/2920312011 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f40fc0b2900 0x7f40fc0b4dc0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f410000ac50 tx=0x7f410000a380 comp rx=0 tx=0).stop 2024-05-03T18:44:23.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:23.195+0000 7f41163af700 1 -- 172.21.15.32:0/2920312011 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4110107700 msgr2=0x7f4110198870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:44:23.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:23.195+0000 7f41163af700 1 --2- 172.21.15.32:0/2920312011 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4110107700 0x7f4110198870 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f410800ef30 tx=0x7f410800c3a0 comp rx=0 tx=0).stop 2024-05-03T18:44:23.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:23.196+0000 7f41163af700 1 -- 172.21.15.32:0/2920312011 shutdown_connections 2024-05-03T18:44:23.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:23.196+0000 7f41163af700 1 --2- 172.21.15.32:0/2920312011 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f40fc0b2900 0x7f40fc0b4dc0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:23.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:23.196+0000 7f41163af700 1 --2- 172.21.15.32:0/2920312011 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4110107700 0x7f4110198870 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:23.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:23.196+0000 7f41163af700 1 --2- 172.21.15.32:0/2920312011 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4110102620 0x7f411019e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:23.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:23.196+0000 7f41163af700 1 --2- 172.21.15.32:0/2920312011 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4110102fd0 0x7f411019ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:23.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:23.196+0000 7f41163af700 1 -- 172.21.15.32:0/2920312011 >> 172.21.15.32:0/2920312011 conn(0x7f4110079440 msgr2=0x7f4110100390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:44:23.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:23.196+0000 7f41163af700 1 -- 172.21.15.32:0/2920312011 shutdown_connections 2024-05-03T18:44:23.197 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:23.196+0000 7f41163af700 1 -- 172.21.15.32:0/2920312011 wait complete. 2024-05-03T18:44:23.198 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 8 2024-05-03T18:44:23.493 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:23 smithi032 ceph-mon[29763]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:44:23.493 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:23 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:23.493 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:23 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:23.493 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:23 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:23.494 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:23 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T18:44:23.494 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:23 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T18:44:23.494 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:23 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:44:23.494 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:23 smithi032 ceph-mon[29763]: Deploying daemon mds.h on smithi045 2024-05-03T18:44:23.494 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:23 smithi032 ceph-mon[29763]: mds.? [v2:172.21.15.113:6836/3677284460,v1:172.21.15.113:6837/3677284460] up:boot 2024-05-03T18:44:23.494 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:23 smithi032 ceph-mon[29763]: fsmap 7 up:standby 2024-05-03T18:44:23.494 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:23 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-03T18:44:23.494 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:23 smithi032 ceph-mon[29763]: from='client.? 172.21.15.32:0/2920312011' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:44:23.747 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:23 smithi045 ceph-mon[33299]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:44:23.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:23 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:23.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:23 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:23.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:23 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:23.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:23 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T18:44:23.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:23 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T18:44:23.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:23 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:44:23.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:23 smithi045 ceph-mon[33299]: Deploying daemon mds.h on smithi045 2024-05-03T18:44:23.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:23 smithi045 ceph-mon[33299]: mds.? [v2:172.21.15.113:6836/3677284460,v1:172.21.15.113:6837/3677284460] up:boot 2024-05-03T18:44:23.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:23 smithi045 ceph-mon[33299]: fsmap 7 up:standby 2024-05-03T18:44:23.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:23 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-03T18:44:23.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:23 smithi045 ceph-mon[33299]: from='client.? 172.21.15.32:0/2920312011' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:44:23.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:23 smithi113 ceph-mon[33540]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:44:23.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:23 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:23.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:23 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:23.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:23 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:23.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:23 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T18:44:23.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:23 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T18:44:23.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:23 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:44:23.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:23 smithi113 ceph-mon[33540]: Deploying daemon mds.h on smithi045 2024-05-03T18:44:23.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:23 smithi113 ceph-mon[33540]: mds.? [v2:172.21.15.113:6836/3677284460,v1:172.21.15.113:6837/3677284460] up:boot 2024-05-03T18:44:23.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:23 smithi113 ceph-mon[33540]: fsmap 7 up:standby 2024-05-03T18:44:23.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:23 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-03T18:44:23.802 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:23 smithi113 ceph-mon[33540]: from='client.? 172.21.15.32:0/2920312011' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:44:24.748 INFO:journalctl@ceph.mds.h.smithi045.stdout:May 03 18:44:24 smithi045 systemd[1]: Starting Ceph mds.h for f500455e-097b-11ef-bc97-c7b262605968... 2024-05-03T18:44:25.080 INFO:journalctl@ceph.mds.h.smithi045.stdout:May 03 18:44:24 smithi045 podman[63458]: 2024-05-03 18:44:24.705792918 +0000 UTC m=+0.018482214 image pull ee4d846202496ba6096593e8e8becc0700a5eb7ebd95536031187f5f337d4012 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd 2024-05-03T18:44:25.081 INFO:journalctl@ceph.mds.h.smithi045.stdout:May 03 18:44:24 smithi045 podman[63458]: 2024-05-03 18:44:24.819418473 +0000 UTC m=+0.132107810 container create 7872d57d7bb32c444723390a305fe3c8c56a4010b5b29e42484396de703442c5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-mds-h, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240430, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe) 2024-05-03T18:44:25.081 INFO:journalctl@ceph.mds.h.smithi045.stdout:May 03 18:44:25 smithi045 podman[63458]: 2024-05-03 18:44:25.005297853 +0000 UTC m=+0.317987151 container init 7872d57d7bb32c444723390a305fe3c8c56a4010b5b29e42484396de703442c5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-mds-h, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240430, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-05-03T18:44:25.081 INFO:journalctl@ceph.mds.h.smithi045.stdout:May 03 18:44:25 smithi045 podman[63458]: 2024-05-03 18:44:25.008797831 +0000 UTC m=+0.321487135 container start 7872d57d7bb32c444723390a305fe3c8c56a4010b5b29e42484396de703442c5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-mds-h, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_CLEAN=True, org.label-schema.build-date=20240430, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS) 2024-05-03T18:44:25.081 INFO:journalctl@ceph.mds.h.smithi045.stdout:May 03 18:44:25 smithi045 ceph-mds[63477]: -- 172.21.15.45:0/2310941206 <== mon.2 v2:172.21.15.45:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557f43d1fa40 con 0x557f43cf2c00 2024-05-03T18:44:25.081 INFO:journalctl@ceph.mds.h.smithi045.stdout:May 03 18:44:25 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-mds-h[63473]: starting mds.h at 2024-05-03T18:44:25.081 INFO:journalctl@ceph.mds.h.smithi045.stdout:May 03 18:44:25 smithi045 bash[63458]: 7872d57d7bb32c444723390a305fe3c8c56a4010b5b29e42484396de703442c5 2024-05-03T18:44:25.442 INFO:journalctl@ceph.mds.h.smithi045.stdout:May 03 18:44:25 smithi045 systemd[1]: Started Ceph mds.h for f500455e-097b-11ef-bc97-c7b262605968. 2024-05-03T18:44:25.747 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:25 smithi045 ceph-mon[33299]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:44:25.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:25 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:25.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:25 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:25.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:25 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:25.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:25 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T18:44:25.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:25 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T18:44:25.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:25 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:44:25.792 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:25 smithi113 ceph-mon[33540]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:44:25.792 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:25 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:25.792 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:25 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:25.792 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:25 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:25.792 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:25 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T18:44:25.792 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:25 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T18:44:25.792 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:25 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:44:25.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:25 smithi032 ceph-mon[29763]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:44:25.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:25 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:25.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:25 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:25.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:25 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:25.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:25 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T18:44:25.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:25 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T18:44:25.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:25 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:44:25.869 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid f500455e-097b-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-03T18:44:26.132 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:44:26.534 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:26 smithi032 ceph-mon[29763]: Deploying daemon mds.i on smithi113 2024-05-03T18:44:26.534 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:26 smithi032 ceph-mon[29763]: mds.? [v2:172.21.15.45:6836/428055392,v1:172.21.15.45:6837/428055392] up:boot 2024-05-03T18:44:26.534 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:26 smithi032 ceph-mon[29763]: fsmap 8 up:standby 2024-05-03T18:44:26.534 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:26 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-03T18:44:26.747 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:26 smithi045 ceph-mon[33299]: Deploying daemon mds.i on smithi113 2024-05-03T18:44:26.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:26 smithi045 ceph-mon[33299]: mds.? [v2:172.21.15.45:6836/428055392,v1:172.21.15.45:6837/428055392] up:boot 2024-05-03T18:44:26.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:26 smithi045 ceph-mon[33299]: fsmap 8 up:standby 2024-05-03T18:44:26.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:26 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-03T18:44:26.773 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:26 smithi113 ceph-mon[33540]: Deploying daemon mds.i on smithi113 2024-05-03T18:44:26.773 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:26 smithi113 ceph-mon[33540]: mds.? [v2:172.21.15.45:6836/428055392,v1:172.21.15.45:6837/428055392] up:boot 2024-05-03T18:44:26.773 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:26 smithi113 ceph-mon[33540]: fsmap 8 up:standby 2024-05-03T18:44:26.773 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:26 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-03T18:44:27.454 INFO:journalctl@ceph.mds.i.smithi113.stdout:May 03 18:44:27 smithi113 systemd[1]: Starting Ceph mds.i for f500455e-097b-11ef-bc97-c7b262605968... 2024-05-03T18:44:27.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:27 smithi032 ceph-mon[29763]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:44:27.706 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:27 smithi113 ceph-mon[33540]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:44:27.706 INFO:journalctl@ceph.mds.i.smithi113.stdout:May 03 18:44:27 smithi113 podman[65208]: 2024-05-03 18:44:27.365566279 +0000 UTC m=+0.018039808 image pull ee4d846202496ba6096593e8e8becc0700a5eb7ebd95536031187f5f337d4012 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd 2024-05-03T18:44:27.706 INFO:journalctl@ceph.mds.i.smithi113.stdout:May 03 18:44:27 smithi113 podman[65208]: 2024-05-03 18:44:27.492420585 +0000 UTC m=+0.144894103 container create e79590a06bad825829089a598f2f37de964a9edb95ace200ee63454647278647 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-mds-i, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_CLEAN=True, org.label-schema.build-date=20240430) 2024-05-03T18:44:27.706 INFO:journalctl@ceph.mds.i.smithi113.stdout:May 03 18:44:27 smithi113 podman[65208]: 2024-05-03 18:44:27.703500005 +0000 UTC m=+0.355973523 container init e79590a06bad825829089a598f2f37de964a9edb95ace200ee63454647278647 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-mds-i, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240430, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-03T18:44:27.706 INFO:journalctl@ceph.mds.i.smithi113.stdout:May 03 18:44:27 smithi113 podman[65208]: 2024-05-03 18:44:27.705981228 +0000 UTC m=+0.358454749 container start e79590a06bad825829089a598f2f37de964a9edb95ace200ee63454647278647 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-mds-i, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.build-date=20240430) 2024-05-03T18:44:27.747 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:27 smithi045 ceph-mon[33299]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:44:27.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:27.778+0000 7f59e4a5b700 1 -- 172.21.15.32:0/1675329351 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59e0103550 msgr2=0x7f59e01039b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:44:27.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:27.778+0000 7f59e4a5b700 1 --2- 172.21.15.32:0/1675329351 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59e0103550 0x7f59e01039b0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f59d4009990 tx=0x7f59d4009ca0 comp rx=0 tx=0).stop 2024-05-03T18:44:27.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:27.778+0000 7f59e4a5b700 1 -- 172.21.15.32:0/1675329351 shutdown_connections 2024-05-03T18:44:27.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:27.778+0000 7f59e4a5b700 1 --2- 172.21.15.32:0/1675329351 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f59e0103ef0 0x7f59e01103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:27.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:27.778+0000 7f59e4a5b700 1 --2- 172.21.15.32:0/1675329351 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59e0108460 0x7f59e0108840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:27.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:27.778+0000 7f59e4a5b700 1 --2- 172.21.15.32:0/1675329351 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59e0103550 0x7f59e01039b0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:27.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:27.778+0000 7f59e4a5b700 1 -- 172.21.15.32:0/1675329351 >> 172.21.15.32:0/1675329351 conn(0x7f59e007a600 msgr2=0x7f59e007aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:44:27.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:27.778+0000 7f59e4a5b700 1 -- 172.21.15.32:0/1675329351 shutdown_connections 2024-05-03T18:44:27.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:27.778+0000 7f59e4a5b700 1 -- 172.21.15.32:0/1675329351 wait complete. 2024-05-03T18:44:27.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:27.779+0000 7f59e4a5b700 1 Processor -- start 2024-05-03T18:44:27.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:27.779+0000 7f59e4a5b700 1 -- start start 2024-05-03T18:44:27.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:27.779+0000 7f59e4a5b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59e0103550 0x7f59e019f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:27.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:27.779+0000 7f59e4a5b700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f59e0103ef0 0x7f59e019fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:27.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:27.779+0000 7f59e4a5b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59e0108460 0x7f59e01999d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:27.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:27.779+0000 7f59e4a5b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f59e0113690 con 0x7f59e0103550 2024-05-03T18:44:27.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:27.779+0000 7f59e4a5b700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f59e0113510 con 0x7f59e0108460 2024-05-03T18:44:27.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:27.779+0000 7f59e4a5b700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f59e0113810 con 0x7f59e0103ef0 2024-05-03T18:44:27.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:27.780+0000 7f59de59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59e0103550 0x7f59e019f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:44:27.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:27.780+0000 7f59de59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59e0103550 0x7f59e019f7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46234/0 (socket says 172.21.15.32:46234) 2024-05-03T18:44:27.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:27.780+0000 7f59de59c700 1 -- 172.21.15.32:0/3532588864 learned_addr learned my addr 172.21.15.32:0/3532588864 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:44:27.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:27.780+0000 7f59ded9d700 1 --2- 172.21.15.32:0/3532588864 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59e0108460 0x7f59e01999d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:44:27.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:27.780+0000 7f59ddd9b700 1 --2- 172.21.15.32:0/3532588864 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f59e0103ef0 0x7f59e019fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:44:27.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:27.780+0000 7f59de59c700 1 -- 172.21.15.32:0/3532588864 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f59e0103ef0 msgr2=0x7f59e019fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:44:27.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:27.780+0000 7f59de59c700 1 --2- 172.21.15.32:0/3532588864 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f59e0103ef0 0x7f59e019fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:27.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:27.780+0000 7f59de59c700 1 -- 172.21.15.32:0/3532588864 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59e0108460 msgr2=0x7f59e01999d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:44:27.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:27.780+0000 7f59de59c700 1 --2- 172.21.15.32:0/3532588864 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59e0108460 0x7f59e01999d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:27.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:27.780+0000 7f59de59c700 1 -- 172.21.15.32:0/3532588864 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59e019a130 con 0x7f59e0103550 2024-05-03T18:44:27.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:27.780+0000 7f59de59c700 1 --2- 172.21.15.32:0/3532588864 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59e0103550 0x7f59e019f7c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f59d000d980 tx=0x7f59d000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:44:27.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:27.780+0000 7f59cf7fe700 1 -- 172.21.15.32:0/3532588864 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59d0017030 con 0x7f59e0103550 2024-05-03T18:44:27.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:27.780+0000 7f59e4a5b700 1 -- 172.21.15.32:0/3532588864 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59e019a420 con 0x7f59e0103550 2024-05-03T18:44:27.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:27.780+0000 7f59cf7fe700 1 -- 172.21.15.32:0/3532588864 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f59d00044d0 con 0x7f59e0103550 2024-05-03T18:44:27.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:27.780+0000 7f59cf7fe700 1 -- 172.21.15.32:0/3532588864 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59d0003970 con 0x7f59e0103550 2024-05-03T18:44:27.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:27.780+0000 7f59e4a5b700 1 -- 172.21.15.32:0/3532588864 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59e01a6690 con 0x7f59e0103550 2024-05-03T18:44:27.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:27.781+0000 7f59e4a5b700 1 -- 172.21.15.32:0/3532588864 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59e006a190 con 0x7f59e0103550 2024-05-03T18:44:27.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:27.783+0000 7f59cf7fe700 1 -- 172.21.15.32:0/3532588864 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f59d001a070 con 0x7f59e0103550 2024-05-03T18:44:27.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:27.783+0000 7f59cf7fe700 1 --2- 172.21.15.32:0/3532588864 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f59c80b2880 0x7f59c80b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:27.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:27.783+0000 7f59ddd9b700 1 --2- 172.21.15.32:0/3532588864 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f59c80b2880 0x7f59c80b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:44:27.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:27.784+0000 7f59cf7fe700 1 -- 172.21.15.32:0/3532588864 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f59d00e1980 con 0x7f59e0103550 2024-05-03T18:44:27.785 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:27.784+0000 7f59ddd9b700 1 --2- 172.21.15.32:0/3532588864 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f59c80b2880 0x7f59c80b4d40 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f59d400b2e0 tx=0x7f59d4005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:44:27.788 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:27.787+0000 7f59cf7fe700 1 -- 172.21.15.32:0/3532588864 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f59d00a9b40 con 0x7f59e0103550 2024-05-03T18:44:28.051 INFO:journalctl@ceph.mds.i.smithi113.stdout:May 03 18:44:27 smithi113 ceph-mds[65227]: -- 172.21.15.113:0/3694938068 <== mon.1 v2:172.21.15.113:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55dc80e75a40 con 0x55dc80e48c00 2024-05-03T18:44:28.051 INFO:journalctl@ceph.mds.i.smithi113.stdout:May 03 18:44:27 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-mds-i[65223]: starting mds.i at 2024-05-03T18:44:28.051 INFO:journalctl@ceph.mds.i.smithi113.stdout:May 03 18:44:27 smithi113 bash[65208]: e79590a06bad825829089a598f2f37de964a9edb95ace200ee63454647278647 2024-05-03T18:44:28.051 INFO:journalctl@ceph.mds.i.smithi113.stdout:May 03 18:44:27 smithi113 systemd[1]: Started Ceph mds.i for f500455e-097b-11ef-bc97-c7b262605968. 2024-05-03T18:44:28.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:28.061+0000 7f59e4a5b700 1 -- 172.21.15.32:0/3532588864 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f59e002e510 con 0x7f59e0103550 2024-05-03T18:44:28.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:28.061+0000 7f59cf7fe700 1 -- 172.21.15.32:0/3532588864 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7171 (secure 0 0 0) 0x7f59d00a9440 con 0x7f59e0103550 2024-05-03T18:44:28.062 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-03T18:44:28.063 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/535599929","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":535599929},{"type":"v1","addr":"172.21.15.32:6835","nonce":535599929}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/3101679868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":3101679868},{"type":"v1","addr":"172.21.15.32:6837","nonce":3101679868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24380,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/1629869435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":1629869435},{"type":"v1","addr":"172.21.15.45:6833","nonce":1629869435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24398,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2997291166","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2997291166},{"type":"v1","addr":"172.21.15.45:6835","nonce":2997291166}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24425,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/428055392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":428055392},{"type":"v1","addr":"172.21.15.45:6837","nonce":428055392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6833/3323558791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":3323558791},{"type":"v1","addr":"172.21.15.113:6833","nonce":3323558791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24463,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/2588738276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2588738276},{"type":"v1","addr":"172.21.15.113:6835","nonce":2588738276}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/3677284460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":3677284460},{"type":"v1","addr":"172.21.15.113:6837","nonce":3677284460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-05-03T18:44:28.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:28.063+0000 7f59e4a5b700 1 -- 172.21.15.32:0/3532588864 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f59c80b2880 msgr2=0x7f59c80b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:44:28.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:28.063+0000 7f59e4a5b700 1 --2- 172.21.15.32:0/3532588864 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f59c80b2880 0x7f59c80b4d40 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f59d400b2e0 tx=0x7f59d4005960 comp rx=0 tx=0).stop 2024-05-03T18:44:28.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:28.063+0000 7f59e4a5b700 1 -- 172.21.15.32:0/3532588864 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59e0103550 msgr2=0x7f59e019f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:44:28.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:28.063+0000 7f59e4a5b700 1 --2- 172.21.15.32:0/3532588864 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59e0103550 0x7f59e019f7c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f59d000d980 tx=0x7f59d000dc90 comp rx=0 tx=0).stop 2024-05-03T18:44:28.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:28.063+0000 7f59e4a5b700 1 -- 172.21.15.32:0/3532588864 shutdown_connections 2024-05-03T18:44:28.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:28.063+0000 7f59e4a5b700 1 --2- 172.21.15.32:0/3532588864 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f59c80b2880 0x7f59c80b4d40 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:28.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:28.063+0000 7f59e4a5b700 1 --2- 172.21.15.32:0/3532588864 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f59e0108460 0x7f59e01999d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:28.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:28.063+0000 7f59e4a5b700 1 --2- 172.21.15.32:0/3532588864 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59e0103550 0x7f59e019f7c0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:28.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:28.064+0000 7f59e4a5b700 1 --2- 172.21.15.32:0/3532588864 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f59e0103ef0 0x7f59e019fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:28.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:28.064+0000 7f59e4a5b700 1 -- 172.21.15.32:0/3532588864 >> 172.21.15.32:0/3532588864 conn(0x7f59e007a600 msgr2=0x7f59e0101640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:44:28.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:28.064+0000 7f59e4a5b700 1 -- 172.21.15.32:0/3532588864 shutdown_connections 2024-05-03T18:44:28.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:28.064+0000 7f59e4a5b700 1 -- 172.21.15.32:0/3532588864 wait complete. 2024-05-03T18:44:28.066 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 9 2024-05-03T18:44:28.969 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:28 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:28.969 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:28 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:28.969 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:28 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:28.970 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:28 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T18:44:28.970 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:28 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T18:44:28.970 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:28 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:44:28.970 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:28 smithi032 ceph-mon[29763]: Deploying daemon mds.g on smithi032 2024-05-03T18:44:28.970 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:28 smithi032 ceph-mon[29763]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:44:28.970 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:28 smithi032 ceph-mon[29763]: from='client.? 172.21.15.32:0/3532588864' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:44:28.970 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:28 smithi032 ceph-mon[29763]: mds.? [v2:172.21.15.113:6838/942050693,v1:172.21.15.113:6839/942050693] up:boot 2024-05-03T18:44:28.970 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:28 smithi032 ceph-mon[29763]: fsmap 9 up:standby 2024-05-03T18:44:28.970 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:28 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-03T18:44:29.248 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:28 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:29.248 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:28 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:29.248 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:28 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:29.248 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:28 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T18:44:29.248 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:28 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T18:44:29.248 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:28 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:44:29.248 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:28 smithi045 ceph-mon[33299]: Deploying daemon mds.g on smithi032 2024-05-03T18:44:29.248 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:28 smithi045 ceph-mon[33299]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:44:29.248 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:28 smithi045 ceph-mon[33299]: from='client.? 172.21.15.32:0/3532588864' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:44:29.248 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:28 smithi045 ceph-mon[33299]: mds.? [v2:172.21.15.113:6838/942050693,v1:172.21.15.113:6839/942050693] up:boot 2024-05-03T18:44:29.248 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:28 smithi045 ceph-mon[33299]: fsmap 9 up:standby 2024-05-03T18:44:29.248 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:28 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-03T18:44:29.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:28 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:29.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:28 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:29.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:28 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:29.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:28 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T18:44:29.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:28 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T18:44:29.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:28 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:44:29.302 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:28 smithi113 ceph-mon[33540]: Deploying daemon mds.g on smithi032 2024-05-03T18:44:29.302 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:28 smithi113 ceph-mon[33540]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:44:29.302 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:28 smithi113 ceph-mon[33540]: from='client.? 172.21.15.32:0/3532588864' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:44:29.302 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:28 smithi113 ceph-mon[33540]: mds.? [v2:172.21.15.113:6838/942050693,v1:172.21.15.113:6839/942050693] up:boot 2024-05-03T18:44:29.302 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:28 smithi113 ceph-mon[33540]: fsmap 9 up:standby 2024-05-03T18:44:29.302 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:28 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-03T18:44:30.569 INFO:journalctl@ceph.mds.g.smithi032.stdout:May 03 18:44:30 smithi032 podman[68966]: 2024-05-03 18:44:30.478473747 +0000 UTC m=+0.120374960 container create 43d6b3ac5db50f25bac20cb6d05a53bd6d45588387b4d51e26f2928c7e9c4455 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-mds-g, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240430, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD) 2024-05-03T18:44:30.725 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid f500455e-097b-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-03T18:44:30.990 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:44:31.012 INFO:journalctl@ceph.mds.g.smithi032.stdout:May 03 18:44:30 smithi032 podman[68966]: 2024-05-03 18:44:30.706030806 +0000 UTC m=+0.347932011 container init 43d6b3ac5db50f25bac20cb6d05a53bd6d45588387b4d51e26f2928c7e9c4455 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-mds-g, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240430, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-05-03T18:44:31.012 INFO:journalctl@ceph.mds.g.smithi032.stdout:May 03 18:44:30 smithi032 podman[68966]: 2024-05-03 18:44:30.708772854 +0000 UTC m=+0.350674058 container start 43d6b3ac5db50f25bac20cb6d05a53bd6d45588387b4d51e26f2928c7e9c4455 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-mds-g, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240430, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-05-03T18:44:31.012 INFO:journalctl@ceph.mds.g.smithi032.stdout:May 03 18:44:30 smithi032 ceph-mds[68985]: -- 172.21.15.32:0/3785394553 <== mon.0 v2:172.21.15.32:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f63bf91a40 con 0x55f63bf64c00 2024-05-03T18:44:31.012 INFO:journalctl@ceph.mds.g.smithi032.stdout:May 03 18:44:30 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-mds-g[68981]: starting mds.g at 2024-05-03T18:44:31.013 INFO:journalctl@ceph.mds.g.smithi032.stdout:May 03 18:44:30 smithi032 bash[68966]: 43d6b3ac5db50f25bac20cb6d05a53bd6d45588387b4d51e26f2928c7e9c4455 2024-05-03T18:44:31.013 INFO:journalctl@ceph.mds.g.smithi032.stdout:May 03 18:44:30 smithi032 systemd[1]: Started Ceph mds.g for f500455e-097b-11ef-bc97-c7b262605968. 2024-05-03T18:44:31.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:31 smithi113 ceph-mon[33540]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:44:31.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:31 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:31.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:31 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:31.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:31 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:31.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:31 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T18:44:31.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:31 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T18:44:31.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:31 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:44:31.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:31 smithi032 ceph-mon[29763]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:44:31.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:31 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:31.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:31 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:31.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:31 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:31.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:31 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T18:44:31.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:31 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T18:44:31.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:31 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:44:31.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:31 smithi045 ceph-mon[33299]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:44:31.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:31 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:31.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:31 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:31.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:31 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:31.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:31 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T18:44:31.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:31 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T18:44:31.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:31 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:44:32.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:32 smithi113 ceph-mon[33540]: Deploying daemon mds.a on smithi032 2024-05-03T18:44:32.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:32 smithi113 ceph-mon[33540]: mds.? [v2:172.21.15.32:6838/3838653921,v1:172.21.15.32:6839/3838653921] up:boot 2024-05-03T18:44:32.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:32 smithi113 ceph-mon[33540]: fsmap 10 up:standby 2024-05-03T18:44:32.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:32 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-03T18:44:32.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:32 smithi032 ceph-mon[29763]: Deploying daemon mds.a on smithi032 2024-05-03T18:44:32.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:32 smithi032 ceph-mon[29763]: mds.? [v2:172.21.15.32:6838/3838653921,v1:172.21.15.32:6839/3838653921] up:boot 2024-05-03T18:44:32.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:32 smithi032 ceph-mon[29763]: fsmap 10 up:standby 2024-05-03T18:44:32.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:32 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-03T18:44:32.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:32 smithi045 ceph-mon[33299]: Deploying daemon mds.a on smithi032 2024-05-03T18:44:32.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:32 smithi045 ceph-mon[33299]: mds.? [v2:172.21.15.32:6838/3838653921,v1:172.21.15.32:6839/3838653921] up:boot 2024-05-03T18:44:32.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:32 smithi045 ceph-mon[33299]: fsmap 10 up:standby 2024-05-03T18:44:32.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:32 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-03T18:44:33.119 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:33.101+0000 7f5971d9d700 1 -- 172.21.15.32:0/1548330082 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f596c06c820 msgr2=0x7f596c1080e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:44:33.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:33.101+0000 7f5971d9d700 1 --2- 172.21.15.32:0/1548330082 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f596c06c820 0x7f596c1080e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f59600099c0 tx=0x7f5960009cd0 comp rx=0 tx=0).stop 2024-05-03T18:44:33.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:33.101+0000 7f5971d9d700 1 -- 172.21.15.32:0/1548330082 shutdown_connections 2024-05-03T18:44:33.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:33.101+0000 7f5971d9d700 1 --2- 172.21.15.32:0/1548330082 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f596c1087c0 0x7f596c1103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:33.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:33.101+0000 7f5971d9d700 1 --2- 172.21.15.32:0/1548330082 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f596c06be70 0x7f596c06c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:33.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:33.101+0000 7f5971d9d700 1 --2- 172.21.15.32:0/1548330082 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f596c06c820 0x7f596c1080e0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:33.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:33.101+0000 7f5971d9d700 1 -- 172.21.15.32:0/1548330082 >> 172.21.15.32:0/1548330082 conn(0x7f596c0783f0 msgr2=0x7f596c078800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:44:33.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:33.101+0000 7f5971d9d700 1 -- 172.21.15.32:0/1548330082 shutdown_connections 2024-05-03T18:44:33.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:33.101+0000 7f5971d9d700 1 -- 172.21.15.32:0/1548330082 wait complete. 2024-05-03T18:44:33.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:33.102+0000 7f5971d9d700 1 Processor -- start 2024-05-03T18:44:33.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:33.102+0000 7f5971d9d700 1 -- start start 2024-05-03T18:44:33.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:33.102+0000 7f5971d9d700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f596c06be70 0x7f596c19f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:33.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:33.102+0000 7f5971d9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f596c06c820 0x7f596c19fe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:33.120 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:33.102+0000 7f5971d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f596c1087c0 0x7f596c199af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:33.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:33.102+0000 7f5971d9d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f596c1136f0 con 0x7f596c1087c0 2024-05-03T18:44:33.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:33.102+0000 7f5971d9d700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f596c113570 con 0x7f596c06c820 2024-05-03T18:44:33.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:33.102+0000 7f5971d9d700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f596c113870 con 0x7f596c06be70 2024-05-03T18:44:33.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:33.103+0000 7f596bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f596c1087c0 0x7f596c199af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:44:33.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:33.103+0000 7f596bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f596c1087c0 0x7f596c199af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46266/0 (socket says 172.21.15.32:46266) 2024-05-03T18:44:33.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:33.103+0000 7f596bfff700 1 -- 172.21.15.32:0/300201872 learned_addr learned my addr 172.21.15.32:0/300201872 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:44:33.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:33.103+0000 7f596b7fe700 1 --2- 172.21.15.32:0/300201872 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f596c06be70 0x7f596c19f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:44:33.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:33.103+0000 7f596affd700 1 --2- 172.21.15.32:0/300201872 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f596c06c820 0x7f596c19fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:44:33.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:33.103+0000 7f596bfff700 1 -- 172.21.15.32:0/300201872 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f596c06be70 msgr2=0x7f596c19f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:44:33.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:33.103+0000 7f596bfff700 1 --2- 172.21.15.32:0/300201872 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f596c06be70 0x7f596c19f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:33.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:33.103+0000 7f596bfff700 1 -- 172.21.15.32:0/300201872 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f596c06c820 msgr2=0x7f596c19fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:44:33.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:33.103+0000 7f596bfff700 1 --2- 172.21.15.32:0/300201872 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f596c06c820 0x7f596c19fe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:33.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:33.103+0000 7f596bfff700 1 -- 172.21.15.32:0/300201872 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f596c19a250 con 0x7f596c1087c0 2024-05-03T18:44:33.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:33.103+0000 7f596affd700 1 --2- 172.21.15.32:0/300201872 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f596c06c820 0x7f596c19fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:44:33.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:33.103+0000 7f596bfff700 1 --2- 172.21.15.32:0/300201872 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f596c1087c0 0x7f596c199af0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f596400b970 tx=0x7f596400bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:44:33.121 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:33.103+0000 7f5968ff9700 1 -- 172.21.15.32:0/300201872 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5964019070 con 0x7f596c1087c0 2024-05-03T18:44:33.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:33.103+0000 7f5971d9d700 1 -- 172.21.15.32:0/300201872 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f596c19a540 con 0x7f596c1087c0 2024-05-03T18:44:33.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:33.103+0000 7f5968ff9700 1 -- 172.21.15.32:0/300201872 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f596400cde0 con 0x7f596c1087c0 2024-05-03T18:44:33.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:33.103+0000 7f5968ff9700 1 -- 172.21.15.32:0/300201872 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5964007370 con 0x7f596c1087c0 2024-05-03T18:44:33.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:33.103+0000 7f5971d9d700 1 -- 172.21.15.32:0/300201872 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f596c1a67b0 con 0x7f596c1087c0 2024-05-03T18:44:33.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:33.104+0000 7f5971d9d700 1 -- 172.21.15.32:0/300201872 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f596c06a190 con 0x7f596c1087c0 2024-05-03T18:44:33.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:33.105+0000 7f5968ff9700 1 -- 172.21.15.32:0/300201872 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f596401c070 con 0x7f596c1087c0 2024-05-03T18:44:33.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:33.106+0000 7f5968ff9700 1 --2- 172.21.15.32:0/300201872 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f59540b2880 0x7f59540b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:33.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:33.106+0000 7f596b7fe700 1 --2- 172.21.15.32:0/300201872 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f59540b2880 0x7f59540b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:44:33.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:33.107+0000 7f5968ff9700 1 -- 172.21.15.32:0/300201872 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f59640e1280 con 0x7f596c1087c0 2024-05-03T18:44:33.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:33.107+0000 7f596b7fe700 1 --2- 172.21.15.32:0/300201872 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f59540b2880 0x7f59540b4d40 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f595c005950 tx=0x7f595c00a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:44:33.122 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:33.111+0000 7f5968ff9700 1 -- 172.21.15.32:0/300201872 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f5964051d70 con 0x7f596c1087c0 2024-05-03T18:44:33.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:33 smithi113 ceph-mon[33540]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:44:33.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:33 smithi032 ceph-mon[29763]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:44:33.383 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:33.382+0000 7f5971d9d700 1 -- 172.21.15.32:0/300201872 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f596c19afd0 con 0x7f596c1087c0 2024-05-03T18:44:33.384 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:33.383+0000 7f5968ff9700 1 -- 172.21.15.32:0/300201872 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8824 (secure 0 0 0) 0x7f59640a9440 con 0x7f596c1087c0 2024-05-03T18:44:33.384 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-03T18:44:33.384 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/535599929","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":535599929},{"type":"v1","addr":"172.21.15.32:6835","nonce":535599929}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/3101679868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":3101679868},{"type":"v1","addr":"172.21.15.32:6837","nonce":3101679868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14610,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/3838653921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":3838653921},{"type":"v1","addr":"172.21.15.32:6839","nonce":3838653921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24380,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/1629869435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":1629869435},{"type":"v1","addr":"172.21.15.45:6833","nonce":1629869435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24398,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2997291166","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2997291166},{"type":"v1","addr":"172.21.15.45:6835","nonce":2997291166}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24425,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/428055392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":428055392},{"type":"v1","addr":"172.21.15.45:6837","nonce":428055392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6833/3323558791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":3323558791},{"type":"v1","addr":"172.21.15.113:6833","nonce":3323558791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24463,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/2588738276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2588738276},{"type":"v1","addr":"172.21.15.113:6835","nonce":2588738276}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/3677284460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":3677284460},{"type":"v1","addr":"172.21.15.113:6837","nonce":3677284460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24478,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6839/942050693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6838","nonce":942050693},{"type":"v1","addr":"172.21.15.113:6839","nonce":942050693}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-05-03T18:44:33.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:33.385+0000 7f5971d9d700 1 -- 172.21.15.32:0/300201872 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f59540b2880 msgr2=0x7f59540b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:44:33.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:33.385+0000 7f5971d9d700 1 --2- 172.21.15.32:0/300201872 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f59540b2880 0x7f59540b4d40 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f595c005950 tx=0x7f595c00a300 comp rx=0 tx=0).stop 2024-05-03T18:44:33.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:33.385+0000 7f5971d9d700 1 -- 172.21.15.32:0/300201872 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f596c1087c0 msgr2=0x7f596c199af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:44:33.386 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:33.385+0000 7f5971d9d700 1 --2- 172.21.15.32:0/300201872 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f596c1087c0 0x7f596c199af0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f596400b970 tx=0x7f596400bc80 comp rx=0 tx=0).stop 2024-05-03T18:44:33.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:33.385+0000 7f5971d9d700 1 -- 172.21.15.32:0/300201872 shutdown_connections 2024-05-03T18:44:33.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:33.385+0000 7f5971d9d700 1 --2- 172.21.15.32:0/300201872 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f59540b2880 0x7f59540b4d40 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:33.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:33.385+0000 7f5971d9d700 1 --2- 172.21.15.32:0/300201872 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f596c1087c0 0x7f596c199af0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:33.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:33.385+0000 7f5971d9d700 1 --2- 172.21.15.32:0/300201872 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f596c06be70 0x7f596c19f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:33.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:33.385+0000 7f5971d9d700 1 --2- 172.21.15.32:0/300201872 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f596c06c820 0x7f596c19fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:33.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:33.385+0000 7f5971d9d700 1 -- 172.21.15.32:0/300201872 >> 172.21.15.32:0/300201872 conn(0x7f596c0783f0 msgr2=0x7f596c1074a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:44:33.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:33.385+0000 7f5971d9d700 1 -- 172.21.15.32:0/300201872 shutdown_connections 2024-05-03T18:44:33.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:33.385+0000 7f5971d9d700 1 -- 172.21.15.32:0/300201872 wait complete. 2024-05-03T18:44:33.387 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 11 2024-05-03T18:44:33.497 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:33 smithi045 ceph-mon[33299]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:44:34.031 INFO:journalctl@ceph.mds.a.smithi032.stdout:May 03 18:44:33 smithi032 systemd[1]: Starting Ceph mds.a for f500455e-097b-11ef-bc97-c7b262605968... 2024-05-03T18:44:34.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:34 smithi113 ceph-mon[33540]: from='client.? 172.21.15.32:0/300201872' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:44:34.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:34 smithi032 ceph-mon[29763]: from='client.? 172.21.15.32:0/300201872' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:44:34.497 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:34 smithi045 ceph-mon[33299]: from='client.? 172.21.15.32:0/300201872' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:44:34.656 INFO:journalctl@ceph.mds.a.smithi032.stdout:May 03 18:44:34 smithi032 podman[69428]: 2024-05-03 18:44:34.267675918 +0000 UTC m=+0.018253495 image pull ee4d846202496ba6096593e8e8becc0700a5eb7ebd95536031187f5f337d4012 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd 2024-05-03T18:44:34.656 INFO:journalctl@ceph.mds.a.smithi032.stdout:May 03 18:44:34 smithi032 podman[69428]: 2024-05-03 18:44:34.370012617 +0000 UTC m=+0.120590193 container create 3a5ed69a830240357db8fb389c47dd24041acc5681b0ffb0a82847d8f074e0e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-mds-a, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240430, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-05-03T18:44:34.656 INFO:journalctl@ceph.mds.a.smithi032.stdout:May 03 18:44:34 smithi032 podman[69428]: 2024-05-03 18:44:34.564011992 +0000 UTC m=+0.314589569 container init 3a5ed69a830240357db8fb389c47dd24041acc5681b0ffb0a82847d8f074e0e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-mds-a, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240430, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-05-03T18:44:34.656 INFO:journalctl@ceph.mds.a.smithi032.stdout:May 03 18:44:34 smithi032 podman[69428]: 2024-05-03 18:44:34.566920153 +0000 UTC m=+0.317497732 container start 3a5ed69a830240357db8fb389c47dd24041acc5681b0ffb0a82847d8f074e0e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-mds-a, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240430, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-05-03T18:44:34.656 INFO:journalctl@ceph.mds.a.smithi032.stdout:May 03 18:44:34 smithi032 ceph-mds[69448]: -- 172.21.15.32:0/1974139193 <== mon.0 v2:172.21.15.32:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564d3db6da40 con 0x564d3db40800 2024-05-03T18:44:34.656 INFO:journalctl@ceph.mds.a.smithi032.stdout:May 03 18:44:34 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-mds-a[69444]: starting mds.a at 2024-05-03T18:44:34.657 INFO:journalctl@ceph.mds.a.smithi032.stdout:May 03 18:44:34 smithi032 bash[69428]: 3a5ed69a830240357db8fb389c47dd24041acc5681b0ffb0a82847d8f074e0e4 2024-05-03T18:44:35.037 INFO:journalctl@ceph.mds.a.smithi032.stdout:May 03 18:44:34 smithi032 systemd[1]: Started Ceph mds.a for f500455e-097b-11ef-bc97-c7b262605968. 2024-05-03T18:44:35.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:35 smithi113 ceph-mon[33540]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:44:35.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:35 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:35.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:35 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:35.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:35 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:35.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:35 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T18:44:35.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:35 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T18:44:35.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:35 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:44:35.313 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:35 smithi032 ceph-mon[29763]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:44:35.313 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:35 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:35.313 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:35 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:35.313 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:35 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:35.313 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:35 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T18:44:35.313 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:35 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T18:44:35.313 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:35 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:44:35.389 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:35 smithi045 ceph-mon[33299]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:44:35.389 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:35 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:35.389 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:35 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:35.389 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:35 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:35.389 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:35 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-03T18:44:35.390 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:35 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-03T18:44:35.390 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:35 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:44:36.172 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid f500455e-097b-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-03T18:44:36.296 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:36 smithi032 ceph-mon[29763]: Deploying daemon mds.b on smithi045 2024-05-03T18:44:36.296 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:36 smithi032 ceph-mon[29763]: mds.? [v2:172.21.15.32:6840/3019648436,v1:172.21.15.32:6841/3019648436] up:boot 2024-05-03T18:44:36.296 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:36 smithi032 ceph-mon[29763]: fsmap 11 up:standby 2024-05-03T18:44:36.296 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:36 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-03T18:44:36.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:36 smithi113 ceph-mon[33540]: Deploying daemon mds.b on smithi045 2024-05-03T18:44:36.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:36 smithi113 ceph-mon[33540]: mds.? [v2:172.21.15.32:6840/3019648436,v1:172.21.15.32:6841/3019648436] up:boot 2024-05-03T18:44:36.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:36 smithi113 ceph-mon[33540]: fsmap 11 up:standby 2024-05-03T18:44:36.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:36 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-03T18:44:36.427 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:44:36.445 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:36 smithi045 ceph-mon[33299]: Deploying daemon mds.b on smithi045 2024-05-03T18:44:36.445 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:36 smithi045 ceph-mon[33299]: mds.? [v2:172.21.15.32:6840/3019648436,v1:172.21.15.32:6841/3019648436] up:boot 2024-05-03T18:44:36.445 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:36 smithi045 ceph-mon[33299]: fsmap 11 up:standby 2024-05-03T18:44:36.445 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:36 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-03T18:44:36.998 INFO:journalctl@ceph.mds.b.smithi045.stdout:May 03 18:44:36 smithi045 systemd[1]: Starting Ceph mds.b for f500455e-097b-11ef-bc97-c7b262605968... 2024-05-03T18:44:37.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:37 smithi113 ceph-mon[33540]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:44:37.304 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:37 smithi045 ceph-mon[33299]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:44:37.304 INFO:journalctl@ceph.mds.b.smithi045.stdout:May 03 18:44:37 smithi045 podman[63774]: 2024-05-03 18:44:37.058504346 +0000 UTC m=+0.018993114 image pull ee4d846202496ba6096593e8e8becc0700a5eb7ebd95536031187f5f337d4012 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd 2024-05-03T18:44:37.304 INFO:journalctl@ceph.mds.b.smithi045.stdout:May 03 18:44:37 smithi045 podman[63774]: 2024-05-03 18:44:37.180049621 +0000 UTC m=+0.140538377 container create f3935e6c79c24fc8bd260414749603f62b715186361832ee0121efba5090ff44 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-mds-b, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240430, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe) 2024-05-03T18:44:37.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:37 smithi032 ceph-mon[29763]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:44:37.748 INFO:journalctl@ceph.mds.b.smithi045.stdout:May 03 18:44:37 smithi045 podman[63774]: 2024-05-03 18:44:37.424167332 +0000 UTC m=+0.384656096 container init f3935e6c79c24fc8bd260414749603f62b715186361832ee0121efba5090ff44 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-mds-b, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, io.buildah.version=1.33.5, org.label-schema.build-date=20240430, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-05-03T18:44:37.748 INFO:journalctl@ceph.mds.b.smithi045.stdout:May 03 18:44:37 smithi045 podman[63774]: 2024-05-03 18:44:37.426905814 +0000 UTC m=+0.387394579 container start f3935e6c79c24fc8bd260414749603f62b715186361832ee0121efba5090ff44 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-mds-b, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240430, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-03T18:44:37.748 INFO:journalctl@ceph.mds.b.smithi045.stdout:May 03 18:44:37 smithi045 ceph-mds[63794]: -- 172.21.15.45:0/3156090301 <== mon.2 v2:172.21.15.45:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556641631a40 con 0x556641605000 2024-05-03T18:44:37.748 INFO:journalctl@ceph.mds.b.smithi045.stdout:May 03 18:44:37 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-mds-b[63790]: starting mds.b at 2024-05-03T18:44:37.748 INFO:journalctl@ceph.mds.b.smithi045.stdout:May 03 18:44:37 smithi045 bash[63774]: f3935e6c79c24fc8bd260414749603f62b715186361832ee0121efba5090ff44 2024-05-03T18:44:37.748 INFO:journalctl@ceph.mds.b.smithi045.stdout:May 03 18:44:37 smithi045 systemd[1]: Started Ceph mds.b for f500455e-097b-11ef-bc97-c7b262605968. 2024-05-03T18:44:37.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:37.993+0000 7fb83fb9c700 1 -- 172.21.15.32:0/1380934983 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8381023a0 msgr2=0x7fb838102820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:44:37.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:37.993+0000 7fb83fb9c700 1 --2- 172.21.15.32:0/1380934983 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8381023a0 0x7fb838102820 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fb828009990 tx=0x7fb828009ca0 comp rx=0 tx=0).stop 2024-05-03T18:44:37.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:37.993+0000 7fb83fb9c700 1 -- 172.21.15.32:0/1380934983 shutdown_connections 2024-05-03T18:44:37.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:37.993+0000 7fb83fb9c700 1 --2- 172.21.15.32:0/1380934983 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb838102d60 0x7fb83810f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:37.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:37.993+0000 7fb83fb9c700 1 --2- 172.21.15.32:0/1380934983 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb838108410 0x7fb8381087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:37.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:37.993+0000 7fb83fb9c700 1 --2- 172.21.15.32:0/1380934983 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8381023a0 0x7fb838102820 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:37.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:37.993+0000 7fb83fb9c700 1 -- 172.21.15.32:0/1380934983 >> 172.21.15.32:0/1380934983 conn(0x7fb838079350 msgr2=0x7fb838079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:44:37.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:37.993+0000 7fb83fb9c700 1 -- 172.21.15.32:0/1380934983 shutdown_connections 2024-05-03T18:44:37.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:37.993+0000 7fb83fb9c700 1 -- 172.21.15.32:0/1380934983 wait complete. 2024-05-03T18:44:37.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:37.993+0000 7fb83fb9c700 1 Processor -- start 2024-05-03T18:44:37.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:37.993+0000 7fb83fb9c700 1 -- start start 2024-05-03T18:44:37.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:37.994+0000 7fb83fb9c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8381023a0 0x7fb8381043f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:37.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:37.994+0000 7fb83fb9c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb838102d60 0x7fb838104930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:37.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:37.994+0000 7fb83fb9c700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb838108410 0x7fb838104e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:37.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:37.994+0000 7fb83fb9c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fb838112560 con 0x7fb8381023a0 2024-05-03T18:44:37.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:37.994+0000 7fb83fb9c700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb8381123e0 con 0x7fb838102d60 2024-05-03T18:44:37.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:37.994+0000 7fb83fb9c700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fb8381126e0 con 0x7fb838108410 2024-05-03T18:44:37.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:37.994+0000 7fb83d938700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8381023a0 0x7fb8381043f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:44:37.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:37.994+0000 7fb83d938700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8381023a0 0x7fb8381043f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54688/0 (socket says 172.21.15.32:54688) 2024-05-03T18:44:37.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:37.994+0000 7fb83d938700 1 -- 172.21.15.32:0/228476363 learned_addr learned my addr 172.21.15.32:0/228476363 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:44:37.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:37.994+0000 7fb83d137700 1 --2- 172.21.15.32:0/228476363 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb838102d60 0x7fb838104930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:44:37.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:37.994+0000 7fb83e139700 1 --2- 172.21.15.32:0/228476363 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb838108410 0x7fb838104e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:44:37.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:37.994+0000 7fb83d938700 1 -- 172.21.15.32:0/228476363 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb838108410 msgr2=0x7fb838104e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:44:37.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:37.994+0000 7fb83d938700 1 --2- 172.21.15.32:0/228476363 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb838108410 0x7fb838104e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:37.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:37.994+0000 7fb83d938700 1 -- 172.21.15.32:0/228476363 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb838102d60 msgr2=0x7fb838104930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:44:37.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:37.994+0000 7fb83d938700 1 --2- 172.21.15.32:0/228476363 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb838102d60 0x7fb838104930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:37.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:37.994+0000 7fb83d938700 1 -- 172.21.15.32:0/228476363 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8381a80d0 con 0x7fb8381023a0 2024-05-03T18:44:37.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:37.994+0000 7fb83d938700 1 --2- 172.21.15.32:0/228476363 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8381023a0 0x7fb8381043f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fb83400e9a0 tx=0x7fb83400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:44:37.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:37.994+0000 7fb82effd700 1 -- 172.21.15.32:0/228476363 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb83400ca10 con 0x7fb8381023a0 2024-05-03T18:44:37.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:37.995+0000 7fb82effd700 1 -- 172.21.15.32:0/228476363 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb83400cbb0 con 0x7fb8381023a0 2024-05-03T18:44:37.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:37.995+0000 7fb83fb9c700 1 -- 172.21.15.32:0/228476363 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8381a83c0 con 0x7fb8381023a0 2024-05-03T18:44:37.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:37.995+0000 7fb82effd700 1 -- 172.21.15.32:0/228476363 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb834018680 con 0x7fb8381023a0 2024-05-03T18:44:37.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:37.995+0000 7fb83fb9c700 1 -- 172.21.15.32:0/228476363 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8381a88d0 con 0x7fb8381023a0 2024-05-03T18:44:37.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:37.996+0000 7fb83fb9c700 1 -- 172.21.15.32:0/228476363 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb83806a190 con 0x7fb8381023a0 2024-05-03T18:44:37.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:37.997+0000 7fb82effd700 1 -- 172.21.15.32:0/228476363 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fb834010430 con 0x7fb8381023a0 2024-05-03T18:44:37.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:37.998+0000 7fb82effd700 1 --2- 172.21.15.32:0/228476363 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fb8240b2880 0x7fb8240b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:37.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:37.998+0000 7fb83d137700 1 --2- 172.21.15.32:0/228476363 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fb8240b2880 0x7fb8240b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:44:38.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:37.998+0000 7fb82effd700 1 -- 172.21.15.32:0/228476363 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7fb8340e2220 con 0x7fb8381023a0 2024-05-03T18:44:38.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:37.998+0000 7fb83d137700 1 --2- 172.21.15.32:0/228476363 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fb8240b2880 0x7fb8240b4d40 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fb82800b2e0 tx=0x7fb828005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:44:38.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:38.001+0000 7fb82effd700 1 -- 172.21.15.32:0/228476363 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fb8340aa340 con 0x7fb8381023a0 2024-05-03T18:44:38.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:38.265+0000 7fb83fb9c700 1 -- 172.21.15.32:0/228476363 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb8381a8b50 con 0x7fb8381023a0 2024-05-03T18:44:38.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:38.266+0000 7fb82effd700 1 -- 172.21.15.32:0/228476363 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10473 (secure 0 0 0) 0x7fb8340b23c0 con 0x7fb8381023a0 2024-05-03T18:44:38.267 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-03T18:44:38.267 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/535599929","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":535599929},{"type":"v1","addr":"172.21.15.32:6835","nonce":535599929}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/3101679868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":3101679868},{"type":"v1","addr":"172.21.15.32:6837","nonce":3101679868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14610,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/3838653921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":3838653921},{"type":"v1","addr":"172.21.15.32:6839","nonce":3838653921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14622,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6841/3019648436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6840","nonce":3019648436},{"type":"v1","addr":"172.21.15.32:6841","nonce":3019648436}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24380,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/1629869435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":1629869435},{"type":"v1","addr":"172.21.15.45:6833","nonce":1629869435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24398,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2997291166","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2997291166},{"type":"v1","addr":"172.21.15.45:6835","nonce":2997291166}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24425,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/428055392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":428055392},{"type":"v1","addr":"172.21.15.45:6837","nonce":428055392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6833/3323558791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":3323558791},{"type":"v1","addr":"172.21.15.113:6833","nonce":3323558791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24446,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/970804040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":970804040},{"type":"v1","addr":"172.21.15.45:6839","nonce":970804040}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":24463,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/2588738276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2588738276},{"type":"v1","addr":"172.21.15.113:6835","nonce":2588738276}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/3677284460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":3677284460},{"type":"v1","addr":"172.21.15.113:6837","nonce":3677284460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24478,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6839/942050693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6838","nonce":942050693},{"type":"v1","addr":"172.21.15.113:6839","nonce":942050693}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-05-03T18:44:38.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:38.268+0000 7fb83fb9c700 1 -- 172.21.15.32:0/228476363 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fb8240b2880 msgr2=0x7fb8240b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:44:38.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:38.268+0000 7fb83fb9c700 1 --2- 172.21.15.32:0/228476363 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fb8240b2880 0x7fb8240b4d40 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fb82800b2e0 tx=0x7fb828005960 comp rx=0 tx=0).stop 2024-05-03T18:44:38.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:38.268+0000 7fb83fb9c700 1 -- 172.21.15.32:0/228476363 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8381023a0 msgr2=0x7fb8381043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:44:38.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:38.268+0000 7fb83fb9c700 1 --2- 172.21.15.32:0/228476363 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8381023a0 0x7fb8381043f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fb83400e9a0 tx=0x7fb83400ecb0 comp rx=0 tx=0).stop 2024-05-03T18:44:38.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:38.268+0000 7fb83fb9c700 1 -- 172.21.15.32:0/228476363 shutdown_connections 2024-05-03T18:44:38.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:38.268+0000 7fb83fb9c700 1 --2- 172.21.15.32:0/228476363 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fb8240b2880 0x7fb8240b4d40 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:38.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:38.268+0000 7fb83fb9c700 1 --2- 172.21.15.32:0/228476363 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb838108410 0x7fb838104e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:38.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:38.268+0000 7fb83fb9c700 1 --2- 172.21.15.32:0/228476363 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8381023a0 0x7fb8381043f0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:38.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:38.268+0000 7fb83fb9c700 1 --2- 172.21.15.32:0/228476363 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb838102d60 0x7fb838104930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:38.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:38.268+0000 7fb83fb9c700 1 -- 172.21.15.32:0/228476363 >> 172.21.15.32:0/228476363 conn(0x7fb838079350 msgr2=0x7fb838106630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:44:38.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:38.268+0000 7fb83fb9c700 1 -- 172.21.15.32:0/228476363 shutdown_connections 2024-05-03T18:44:38.270 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:38.268+0000 7fb83fb9c700 1 -- 172.21.15.32:0/228476363 wait complete. 2024-05-03T18:44:38.270 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 13 2024-05-03T18:44:38.664 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:38 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:38.664 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:38 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:38.665 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:38 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:38.665 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:38 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:38.665 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:38 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:38.665 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:38 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:44:38.665 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:38 smithi045 ceph-mon[33299]: mds.? [v2:172.21.15.45:6838/970804040,v1:172.21.15.45:6839/970804040] up:boot 2024-05-03T18:44:38.665 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:38 smithi045 ceph-mon[33299]: fsmap 12 up:standby 2024-05-03T18:44:38.665 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:38 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-03T18:44:38.665 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:38 smithi045 ceph-mon[33299]: from='client.? 172.21.15.32:0/228476363' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:44:38.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:38 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:38.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:38 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:38.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:38 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:38.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:38 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:38.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:38 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:38.819 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:38 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:44:38.819 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:38 smithi032 ceph-mon[29763]: mds.? [v2:172.21.15.45:6838/970804040,v1:172.21.15.45:6839/970804040] up:boot 2024-05-03T18:44:38.819 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:38 smithi032 ceph-mon[29763]: fsmap 12 up:standby 2024-05-03T18:44:38.819 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:38 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-03T18:44:38.819 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:38 smithi032 ceph-mon[29763]: from='client.? 172.21.15.32:0/228476363' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:44:38.850 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid f500455e-097b-11ef-bc97-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-05-03T18:44:39.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:38 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:39.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:38 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:39.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:38 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:39.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:38 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:39.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:38 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:39.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:38 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:44:39.052 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:38 smithi113 ceph-mon[33540]: mds.? [v2:172.21.15.45:6838/970804040,v1:172.21.15.45:6839/970804040] up:boot 2024-05-03T18:44:39.052 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:38 smithi113 ceph-mon[33540]: fsmap 12 up:standby 2024-05-03T18:44:39.052 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:38 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-03T18:44:39.052 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:38 smithi113 ceph-mon[33540]: from='client.? 172.21.15.32:0/228476363' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:44:39.623 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:44:39.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:39 smithi032 ceph-mon[29763]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:44:39.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:39 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:39.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:39 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:39.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:39 smithi045 ceph-mon[33299]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:44:39.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:39 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:39.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:39 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:40.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:39 smithi113 ceph-mon[33540]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:44:40.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:39 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:40.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:39 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:40.922 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:40 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:40.922 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:40 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:40.922 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:40 smithi032 ceph-mon[29763]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:44:40.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:40 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:40.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:40 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:40.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:40 smithi045 ceph-mon[33299]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:44:41.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:40 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:41.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:40 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:41.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:40 smithi113 ceph-mon[33540]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:44:41.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:41.147+0000 7fb432e6f700 1 -- 172.21.15.32:0/4036900262 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb42c102d60 msgr2=0x7fb42c10f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:44:41.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:41.147+0000 7fb432e6f700 1 --2- 172.21.15.32:0/4036900262 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb42c102d60 0x7fb42c10f250 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb42400b0a0 tx=0x7fb42400b3b0 comp rx=0 tx=0).stop 2024-05-03T18:44:41.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:41.147+0000 7fb432e6f700 1 -- 172.21.15.32:0/4036900262 shutdown_connections 2024-05-03T18:44:41.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:41.147+0000 7fb432e6f700 1 --2- 172.21.15.32:0/4036900262 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb42c102d60 0x7fb42c10f250 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:41.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:41.147+0000 7fb432e6f700 1 --2- 172.21.15.32:0/4036900262 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb42c108410 0x7fb42c1087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:41.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:41.147+0000 7fb432e6f700 1 --2- 172.21.15.32:0/4036900262 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb42c1023a0 0x7fb42c102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:41.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:41.147+0000 7fb432e6f700 1 -- 172.21.15.32:0/4036900262 >> 172.21.15.32:0/4036900262 conn(0x7fb42c079350 msgr2=0x7fb42c079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:44:41.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:41.147+0000 7fb432e6f700 1 -- 172.21.15.32:0/4036900262 shutdown_connections 2024-05-03T18:44:41.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:41.147+0000 7fb432e6f700 1 -- 172.21.15.32:0/4036900262 wait complete. 2024-05-03T18:44:41.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:41.148+0000 7fb432e6f700 1 Processor -- start 2024-05-03T18:44:41.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:41.148+0000 7fb432e6f700 1 -- start start 2024-05-03T18:44:41.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:41.148+0000 7fb432e6f700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb42c1023a0 0x7fb42c104080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:41.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:41.148+0000 7fb432e6f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb42c102d60 0x7fb42c1045c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:41.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:41.148+0000 7fb432e6f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb42c108410 0x7fb42c104b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:41.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:41.148+0000 7fb432e6f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fb42c1125b0 con 0x7fb42c108410 2024-05-03T18:44:41.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:41.148+0000 7fb432e6f700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb42c112430 con 0x7fb42c102d60 2024-05-03T18:44:41.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:41.148+0000 7fb432e6f700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fb42c112730 con 0x7fb42c1023a0 2024-05-03T18:44:41.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:41.148+0000 7fb43140c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb42c108410 0x7fb42c104b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:44:41.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:41.148+0000 7fb43140c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb42c108410 0x7fb42c104b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54708/0 (socket says 172.21.15.32:54708) 2024-05-03T18:44:41.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:41.148+0000 7fb43140c700 1 -- 172.21.15.32:0/2250646395 learned_addr learned my addr 172.21.15.32:0/2250646395 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:44:41.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:41.148+0000 7fb42bfff700 1 --2- 172.21.15.32:0/2250646395 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb42c102d60 0x7fb42c1045c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:44:41.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:41.148+0000 7fb430c0b700 1 --2- 172.21.15.32:0/2250646395 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb42c1023a0 0x7fb42c104080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:44:41.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:41.148+0000 7fb43140c700 1 -- 172.21.15.32:0/2250646395 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb42c1023a0 msgr2=0x7fb42c104080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:44:41.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:41.148+0000 7fb43140c700 1 --2- 172.21.15.32:0/2250646395 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb42c1023a0 0x7fb42c104080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:41.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:41.148+0000 7fb43140c700 1 -- 172.21.15.32:0/2250646395 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb42c102d60 msgr2=0x7fb42c1045c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:44:41.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:41.148+0000 7fb43140c700 1 --2- 172.21.15.32:0/2250646395 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb42c102d60 0x7fb42c1045c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:41.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:41.148+0000 7fb43140c700 1 -- 172.21.15.32:0/2250646395 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb42c1a8000 con 0x7fb42c108410 2024-05-03T18:44:41.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:41.148+0000 7fb430c0b700 1 --2- 172.21.15.32:0/2250646395 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb42c1023a0 0x7fb42c104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:44:41.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:41.148+0000 7fb42bfff700 1 --2- 172.21.15.32:0/2250646395 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb42c102d60 0x7fb42c1045c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:44:41.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:41.149+0000 7fb43140c700 1 --2- 172.21.15.32:0/2250646395 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb42c108410 0x7fb42c104b00 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fb4240062a0 tx=0x7fb424004ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:44:41.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:41.149+0000 7fb429ffb700 1 -- 172.21.15.32:0/2250646395 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb424021070 con 0x7fb42c108410 2024-05-03T18:44:41.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:41.149+0000 7fb429ffb700 1 -- 172.21.15.32:0/2250646395 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb424007d00 con 0x7fb42c108410 2024-05-03T18:44:41.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:41.149+0000 7fb432e6f700 1 -- 172.21.15.32:0/2250646395 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb42c1a8290 con 0x7fb42c108410 2024-05-03T18:44:41.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:41.149+0000 7fb429ffb700 1 -- 172.21.15.32:0/2250646395 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb42401d450 con 0x7fb42c108410 2024-05-03T18:44:41.152 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:41.149+0000 7fb432e6f700 1 -- 172.21.15.32:0/2250646395 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb42c1a8790 con 0x7fb42c108410 2024-05-03T18:44:41.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:41.150+0000 7fb432e6f700 1 -- 172.21.15.32:0/2250646395 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb42c06a190 con 0x7fb42c108410 2024-05-03T18:44:41.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:41.151+0000 7fb429ffb700 1 -- 172.21.15.32:0/2250646395 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fb424004020 con 0x7fb42c108410 2024-05-03T18:44:41.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:41.152+0000 7fb429ffb700 1 --2- 172.21.15.32:0/2250646395 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fb4140b2900 0x7fb4140b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:41.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:41.152+0000 7fb430c0b700 1 --2- 172.21.15.32:0/2250646395 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fb4140b2900 0x7fb4140b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:44:41.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:41.152+0000 7fb429ffb700 1 -- 172.21.15.32:0/2250646395 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7fb42400f020 con 0x7fb42c108410 2024-05-03T18:44:41.154 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:41.152+0000 7fb430c0b700 1 --2- 172.21.15.32:0/2250646395 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fb4140b2900 0x7fb4140b4dc0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fb41c005fd0 tx=0x7fb41c005e20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:44:41.157 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:41.157+0000 7fb429ffb700 1 -- 172.21.15.32:0/2250646395 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fb42402a020 con 0x7fb42c108410 2024-05-03T18:44:41.395 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:41.394+0000 7fb432e6f700 1 -- 172.21.15.32:0/2250646395 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fb42c1a8a60 con 0x7fb42c108410 2024-05-03T18:44:41.396 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:41.395+0000 7fb429ffb700 1 -- 172.21.15.32:0/2250646395 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v71) ==== 84+0+65 (secure 0 0 0) 0x7fb4240ab660 con 0x7fb42c108410 2024-05-03T18:44:41.396 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-03T18:44:41.396 INFO:teuthology.orchestra.run.smithi032.stdout:[ 2024-05-03T18:44:41.396 INFO:teuthology.orchestra.run.smithi032.stdout: { 2024-05-03T18:44:41.396 INFO:teuthology.orchestra.run.smithi032.stdout: "poolnum": 1, 2024-05-03T18:44:41.396 INFO:teuthology.orchestra.run.smithi032.stdout: "poolname": ".mgr" 2024-05-03T18:44:41.396 INFO:teuthology.orchestra.run.smithi032.stdout: } 2024-05-03T18:44:41.396 INFO:teuthology.orchestra.run.smithi032.stdout:] 2024-05-03T18:44:41.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:41.397+0000 7fb432e6f700 1 -- 172.21.15.32:0/2250646395 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fb4140b2900 msgr2=0x7fb4140b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:44:41.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:41.397+0000 7fb432e6f700 1 --2- 172.21.15.32:0/2250646395 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fb4140b2900 0x7fb4140b4dc0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fb41c005fd0 tx=0x7fb41c005e20 comp rx=0 tx=0).stop 2024-05-03T18:44:41.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:41.397+0000 7fb432e6f700 1 -- 172.21.15.32:0/2250646395 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb42c108410 msgr2=0x7fb42c104b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:44:41.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:41.397+0000 7fb432e6f700 1 --2- 172.21.15.32:0/2250646395 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb42c108410 0x7fb42c104b00 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fb4240062a0 tx=0x7fb424004ed0 comp rx=0 tx=0).stop 2024-05-03T18:44:41.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:41.397+0000 7fb432e6f700 1 -- 172.21.15.32:0/2250646395 shutdown_connections 2024-05-03T18:44:41.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:41.397+0000 7fb432e6f700 1 --2- 172.21.15.32:0/2250646395 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fb4140b2900 0x7fb4140b4dc0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:41.398 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:41.397+0000 7fb432e6f700 1 --2- 172.21.15.32:0/2250646395 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb42c108410 0x7fb42c104b00 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:41.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:41.397+0000 7fb432e6f700 1 --2- 172.21.15.32:0/2250646395 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb42c1023a0 0x7fb42c104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:41.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:41.397+0000 7fb432e6f700 1 --2- 172.21.15.32:0/2250646395 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb42c102d60 0x7fb42c1045c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:41.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:41.397+0000 7fb432e6f700 1 -- 172.21.15.32:0/2250646395 >> 172.21.15.32:0/2250646395 conn(0x7fb42c079350 msgr2=0x7fb42c1015e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:44:41.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:41.397+0000 7fb432e6f700 1 -- 172.21.15.32:0/2250646395 shutdown_connections 2024-05-03T18:44:41.399 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:41.397+0000 7fb432e6f700 1 -- 172.21.15.32:0/2250646395 wait complete. 2024-05-03T18:44:41.986 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-03T18:44:41.986 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid f500455e-097b-11ef-bc97-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-03T18:44:42.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:41 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:42.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:41 smithi032 ceph-mon[29763]: from='client.? 172.21.15.32:0/2250646395' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-03T18:44:42.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:41 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:42.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:41 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:42.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:41 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:44:42.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:41 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:44:42.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:41 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:42.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:41 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:44:42.247 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:41 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:42.248 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:41 smithi045 ceph-mon[33299]: from='client.? 172.21.15.32:0/2250646395' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-03T18:44:42.248 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:41 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:42.248 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:41 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:42.248 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:41 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:44:42.248 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:41 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:44:42.248 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:41 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:42.248 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:41 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:44:42.248 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:44:42.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:41 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:42.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:41 smithi113 ceph-mon[33540]: from='client.? 172.21.15.32:0/2250646395' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-03T18:44:42.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:41 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:42.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:41 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:42.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:41 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:44:42.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:41 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:44:42.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:41 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:42.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:41 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:44:43.069 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:42 smithi032 ceph-mon[29763]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:44:43.247 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:42 smithi045 ceph-mon[33299]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:44:43.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:42 smithi113 ceph-mon[33540]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:44:44.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:44 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:44.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:44 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:44.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:44 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:44.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:44 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:44.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:44.361+0000 7fe79495b700 1 -- 172.21.15.32:0/2733139109 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe790102fe0 msgr2=0x7fe790106fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:44:44.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:44.361+0000 7fe79495b700 1 --2- 172.21.15.32:0/2733139109 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe790102fe0 0x7fe790106fe0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fe7840099c0 tx=0x7fe784009cd0 comp rx=0 tx=0).stop 2024-05-03T18:44:44.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:44.362+0000 7fe79495b700 1 -- 172.21.15.32:0/2733139109 shutdown_connections 2024-05-03T18:44:44.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:44.362+0000 7fe79495b700 1 --2- 172.21.15.32:0/2733139109 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe7901076c0 0x7fe79010f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:44.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:44.362+0000 7fe79495b700 1 --2- 172.21.15.32:0/2733139109 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe790102630 0x7fe790102a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:44.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:44.362+0000 7fe79495b700 1 --2- 172.21.15.32:0/2733139109 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe790102fe0 0x7fe790106fe0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:44.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:44.362+0000 7fe79495b700 1 -- 172.21.15.32:0/2733139109 >> 172.21.15.32:0/2733139109 conn(0x7fe790079350 msgr2=0x7fe790079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:44:44.370 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:44.362+0000 7fe79495b700 1 -- 172.21.15.32:0/2733139109 shutdown_connections 2024-05-03T18:44:44.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:44.362+0000 7fe79495b700 1 -- 172.21.15.32:0/2733139109 wait complete. 2024-05-03T18:44:44.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:44.362+0000 7fe79495b700 1 Processor -- start 2024-05-03T18:44:44.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:44.363+0000 7fe79495b700 1 -- start start 2024-05-03T18:44:44.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:44.363+0000 7fe79495b700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe790102630 0x7fe79019f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:44.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:44.363+0000 7fe79495b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe790102fe0 0x7fe79019fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:44.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:44.363+0000 7fe79495b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe7901076c0 0x7fe790199ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:44.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:44.363+0000 7fe79495b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe790112550 con 0x7fe790102fe0 2024-05-03T18:44:44.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:44.363+0000 7fe79495b700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe7901123d0 con 0x7fe7901076c0 2024-05-03T18:44:44.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:44.363+0000 7fe79495b700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fe7901126d0 con 0x7fe790102630 2024-05-03T18:44:44.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:44.363+0000 7fe78dd9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe790102fe0 0x7fe79019fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:44:44.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:44.363+0000 7fe78dd9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe790102fe0 0x7fe79019fdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58544/0 (socket says 172.21.15.32:58544) 2024-05-03T18:44:44.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:44.363+0000 7fe78dd9b700 1 -- 172.21.15.32:0/4230770183 learned_addr learned my addr 172.21.15.32:0/4230770183 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:44:44.371 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:44.363+0000 7fe78e59c700 1 --2- 172.21.15.32:0/4230770183 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe790102630 0x7fe79019f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:44:44.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:44.363+0000 7fe78ed9d700 1 --2- 172.21.15.32:0/4230770183 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe7901076c0 0x7fe790199ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:44:44.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:44.363+0000 7fe78dd9b700 1 -- 172.21.15.32:0/4230770183 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe790102630 msgr2=0x7fe79019f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:44:44.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:44.363+0000 7fe78dd9b700 1 --2- 172.21.15.32:0/4230770183 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe790102630 0x7fe79019f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:44.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:44.363+0000 7fe78dd9b700 1 -- 172.21.15.32:0/4230770183 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe7901076c0 msgr2=0x7fe790199ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:44:44.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:44.363+0000 7fe78dd9b700 1 --2- 172.21.15.32:0/4230770183 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe7901076c0 0x7fe790199ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:44.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:44.363+0000 7fe78dd9b700 1 -- 172.21.15.32:0/4230770183 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe79019a220 con 0x7fe790102fe0 2024-05-03T18:44:44.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:44.363+0000 7fe78dd9b700 1 --2- 172.21.15.32:0/4230770183 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe790102fe0 0x7fe79019fdf0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fe784006010 tx=0x7fe7840047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:44:44.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:44.363+0000 7fe77f7fe700 1 -- 172.21.15.32:0/4230770183 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe784027070 con 0x7fe790102fe0 2024-05-03T18:44:44.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:44.364+0000 7fe77f7fe700 1 -- 172.21.15.32:0/4230770183 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe784003f20 con 0x7fe790102fe0 2024-05-03T18:44:44.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:44.364+0000 7fe79495b700 1 -- 172.21.15.32:0/4230770183 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe79019a4b0 con 0x7fe790102fe0 2024-05-03T18:44:44.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:44.364+0000 7fe77f7fe700 1 -- 172.21.15.32:0/4230770183 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe784022cd0 con 0x7fe790102fe0 2024-05-03T18:44:44.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:44.364+0000 7fe79495b700 1 -- 172.21.15.32:0/4230770183 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7901a6780 con 0x7fe790102fe0 2024-05-03T18:44:44.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:44.364+0000 7fe79495b700 1 -- 172.21.15.32:0/4230770183 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe790103ac0 con 0x7fe790102fe0 2024-05-03T18:44:44.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:44.366+0000 7fe77f7fe700 1 -- 172.21.15.32:0/4230770183 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fe78402a070 con 0x7fe790102fe0 2024-05-03T18:44:44.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:44.367+0000 7fe77f7fe700 1 --2- 172.21.15.32:0/4230770183 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fe7780b2880 0x7fe7780b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:44.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:44.367+0000 7fe78e59c700 1 --2- 172.21.15.32:0/4230770183 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fe7780b2880 0x7fe7780b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:44:44.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:44.367+0000 7fe77f7fe700 1 -- 172.21.15.32:0/4230770183 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7fe7840e2ff0 con 0x7fe790102fe0 2024-05-03T18:44:44.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:44.367+0000 7fe78e59c700 1 --2- 172.21.15.32:0/4230770183 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fe7780b2880 0x7fe7780b4d40 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fe790107300 tx=0x7fe78000b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:44:44.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:44.370+0000 7fe77f7fe700 1 -- 172.21.15.32:0/4230770183 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fe784031070 con 0x7fe790102fe0 2024-05-03T18:44:44.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:44 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:44.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:44 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:44.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:44 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:44.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:44 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:44.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:44.603+0000 7fe79495b700 1 -- 172.21.15.32:0/4230770183 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fe79006a190 con 0x7fe790102fe0 2024-05-03T18:44:44.747 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:44 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:44.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:44 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:44.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:44 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:44.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:44 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:45.387 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:45.386+0000 7fe77f7fe700 1 -- 172.21.15.32:0/4230770183 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v72) ==== 138+0+0 (secure 0 0 0) 0x7fe784013070 con 0x7fe790102fe0 2024-05-03T18:44:45.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:45.390+0000 7fe79495b700 1 -- 172.21.15.32:0/4230770183 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fe7780b2880 msgr2=0x7fe7780b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:44:45.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:45.390+0000 7fe79495b700 1 --2- 172.21.15.32:0/4230770183 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fe7780b2880 0x7fe7780b4d40 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fe790107300 tx=0x7fe78000b3c0 comp rx=0 tx=0).stop 2024-05-03T18:44:45.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:45.390+0000 7fe79495b700 1 -- 172.21.15.32:0/4230770183 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe790102fe0 msgr2=0x7fe79019fdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:44:45.391 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:45.390+0000 7fe79495b700 1 --2- 172.21.15.32:0/4230770183 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe790102fe0 0x7fe79019fdf0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fe784006010 tx=0x7fe7840047b0 comp rx=0 tx=0).stop 2024-05-03T18:44:45.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:45.390+0000 7fe79495b700 1 -- 172.21.15.32:0/4230770183 shutdown_connections 2024-05-03T18:44:45.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:45.390+0000 7fe79495b700 1 --2- 172.21.15.32:0/4230770183 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fe7780b2880 0x7fe7780b4d40 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:45.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:45.390+0000 7fe79495b700 1 --2- 172.21.15.32:0/4230770183 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe7901076c0 0x7fe790199ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:45.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:45.390+0000 7fe79495b700 1 --2- 172.21.15.32:0/4230770183 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe790102630 0x7fe79019f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:45.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:45.390+0000 7fe79495b700 1 --2- 172.21.15.32:0/4230770183 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe790102fe0 0x7fe79019fdf0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:45.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:45.390+0000 7fe79495b700 1 -- 172.21.15.32:0/4230770183 >> 172.21.15.32:0/4230770183 conn(0x7fe790079350 msgr2=0x7fe790106440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:44:45.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:45.391+0000 7fe79495b700 1 -- 172.21.15.32:0/4230770183 shutdown_connections 2024-05-03T18:44:45.392 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:45.391+0000 7fe79495b700 1 -- 172.21.15.32:0/4230770183 wait complete. 2024-05-03T18:44:45.394 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_metadata' created 2024-05-03T18:44:45.724 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:45 smithi032 ceph-mon[29763]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:44:45.724 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:45 smithi032 ceph-mon[29763]: from='client.? 172.21.15.32:0/4230770183' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-03T18:44:45.724 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:45 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:45.724 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:45 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:45.724 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:45 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:44:45.724 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:45 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:44:45.725 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:45 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:45.747 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:45 smithi045 ceph-mon[33299]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:44:45.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:45 smithi045 ceph-mon[33299]: from='client.? 172.21.15.32:0/4230770183' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-03T18:44:45.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:45 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:45.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:45 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:45.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:45 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:44:45.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:45 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:44:45.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:45 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:45.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:45 smithi113 ceph-mon[33540]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:44:45.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:45 smithi113 ceph-mon[33540]: from='client.? 172.21.15.32:0/4230770183' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-03T18:44:45.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:45 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:45.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:45 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:45.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:45 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:44:45.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:45 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:44:45.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:45 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:44:46.059 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid f500455e-097b-11ef-bc97-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-03T18:44:46.312 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:44:46.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:46 smithi032 ceph-mon[29763]: from='client.? 172.21.15.32:0/4230770183' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-03T18:44:46.637 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:46 smithi032 ceph-mon[29763]: osdmap e72: 12 total, 12 up, 12 in 2024-05-03T18:44:46.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:46 smithi045 ceph-mon[33299]: from='client.? 172.21.15.32:0/4230770183' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-03T18:44:46.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:46 smithi045 ceph-mon[33299]: osdmap e72: 12 total, 12 up, 12 in 2024-05-03T18:44:46.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:46 smithi113 ceph-mon[33540]: from='client.? 172.21.15.32:0/4230770183' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-03T18:44:46.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:46 smithi113 ceph-mon[33540]: osdmap e72: 12 total, 12 up, 12 in 2024-05-03T18:44:47.737 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:47 smithi032 ceph-mon[29763]: pgmap v272: 33 pgs: 5 creating+peering, 1 active+clean, 27 unknown; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:44:47.737 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:47 smithi032 ceph-mon[29763]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-03T18:44:47.737 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:47 smithi032 ceph-mon[29763]: osdmap e73: 12 total, 12 up, 12 in 2024-05-03T18:44:47.747 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:47 smithi045 ceph-mon[33299]: pgmap v272: 33 pgs: 5 creating+peering, 1 active+clean, 27 unknown; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:44:47.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:47 smithi045 ceph-mon[33299]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-03T18:44:47.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:47 smithi045 ceph-mon[33299]: osdmap e73: 12 total, 12 up, 12 in 2024-05-03T18:44:47.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:47 smithi113 ceph-mon[33540]: pgmap v272: 33 pgs: 5 creating+peering, 1 active+clean, 27 unknown; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:44:47.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:47 smithi113 ceph-mon[33540]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-03T18:44:47.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:47 smithi113 ceph-mon[33540]: osdmap e73: 12 total, 12 up, 12 in 2024-05-03T18:44:47.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:47.850+0000 7ff595d3b700 1 -- 172.21.15.32:0/3537328441 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff5901019f0 msgr2=0x7ff590101e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:44:47.851 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:47.850+0000 7ff595d3b700 1 --2- 172.21.15.32:0/3537328441 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff5901019f0 0x7ff590101e70 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff5840099c0 tx=0x7ff584009cd0 comp rx=0 tx=0).stop 2024-05-03T18:44:47.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:47.850+0000 7ff595d3b700 1 -- 172.21.15.32:0/3537328441 shutdown_connections 2024-05-03T18:44:47.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:47.850+0000 7ff595d3b700 1 --2- 172.21.15.32:0/3537328441 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff5901023b0 0x7ff590110340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:47.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:47.850+0000 7ff595d3b700 1 --2- 172.21.15.32:0/3537328441 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff590109570 0x7ff590109950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:47.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:47.850+0000 7ff595d3b700 1 --2- 172.21.15.32:0/3537328441 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff5901019f0 0x7ff590101e70 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:47.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:47.850+0000 7ff595d3b700 1 -- 172.21.15.32:0/3537328441 >> 172.21.15.32:0/3537328441 conn(0x7ff5900789a0 msgr2=0x7ff5900ffd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:44:47.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:47.850+0000 7ff595d3b700 1 -- 172.21.15.32:0/3537328441 shutdown_connections 2024-05-03T18:44:47.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:47.850+0000 7ff595d3b700 1 -- 172.21.15.32:0/3537328441 wait complete. 2024-05-03T18:44:47.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:47.850+0000 7ff595d3b700 1 Processor -- start 2024-05-03T18:44:47.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:47.850+0000 7ff595d3b700 1 -- start start 2024-05-03T18:44:47.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:47.851+0000 7ff595d3b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff5901019f0 0x7ff59019bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:47.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:47.851+0000 7ff595d3b700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff5901023b0 0x7ff59019c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:47.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:47.851+0000 7ff595d3b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff590109570 0x7ff5901a0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:47.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:47.851+0000 7ff595d3b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff590113730 con 0x7ff5901019f0 2024-05-03T18:44:47.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:47.851+0000 7ff595d3b700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff5901135b0 con 0x7ff590109570 2024-05-03T18:44:47.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:47.851+0000 7ff595d3b700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7ff5901138b0 con 0x7ff5901023b0 2024-05-03T18:44:47.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:47.851+0000 7ff58f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff5901019f0 0x7ff59019bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:44:47.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:47.851+0000 7ff58f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff5901019f0 0x7ff59019bdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58564/0 (socket says 172.21.15.32:58564) 2024-05-03T18:44:47.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:47.851+0000 7ff58f7fe700 1 -- 172.21.15.32:0/128421764 learned_addr learned my addr 172.21.15.32:0/128421764 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:44:47.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:47.851+0000 7ff58effd700 1 --2- 172.21.15.32:0/128421764 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff5901023b0 0x7ff59019c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:44:47.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:47.851+0000 7ff58ffff700 1 --2- 172.21.15.32:0/128421764 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff590109570 0x7ff5901a0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:44:47.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:47.851+0000 7ff58f7fe700 1 -- 172.21.15.32:0/128421764 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff5901023b0 msgr2=0x7ff59019c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:44:47.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:47.851+0000 7ff58f7fe700 1 --2- 172.21.15.32:0/128421764 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff5901023b0 0x7ff59019c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:47.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:47.851+0000 7ff58f7fe700 1 -- 172.21.15.32:0/128421764 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff590109570 msgr2=0x7ff5901a0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:44:47.854 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:47.851+0000 7ff58f7fe700 1 --2- 172.21.15.32:0/128421764 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff590109570 0x7ff5901a0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:47.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:47.851+0000 7ff58f7fe700 1 -- 172.21.15.32:0/128421764 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5901a0dd0 con 0x7ff5901019f0 2024-05-03T18:44:47.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:47.851+0000 7ff58ffff700 1 --2- 172.21.15.32:0/128421764 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff590109570 0x7ff5901a0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T18:44:47.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:47.851+0000 7ff58f7fe700 1 --2- 172.21.15.32:0/128421764 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff5901019f0 0x7ff59019bdc0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ff58000c920 tx=0x7ff58000cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:44:47.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:47.852+0000 7ff58cff9700 1 -- 172.21.15.32:0/128421764 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff580007b00 con 0x7ff5901019f0 2024-05-03T18:44:47.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:47.852+0000 7ff595d3b700 1 -- 172.21.15.32:0/128421764 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5901a10c0 con 0x7ff5901019f0 2024-05-03T18:44:47.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:47.852+0000 7ff58cff9700 1 -- 172.21.15.32:0/128421764 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff580004ce0 con 0x7ff5901019f0 2024-05-03T18:44:47.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:47.852+0000 7ff58cff9700 1 -- 172.21.15.32:0/128421764 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff580005550 con 0x7ff5901019f0 2024-05-03T18:44:47.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:47.852+0000 7ff595d3b700 1 -- 172.21.15.32:0/128421764 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5901a8940 con 0x7ff5901019f0 2024-05-03T18:44:47.855 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:47.853+0000 7ff595d3b700 1 -- 172.21.15.32:0/128421764 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff59006a190 con 0x7ff5901019f0 2024-05-03T18:44:47.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:47.854+0000 7ff58cff9700 1 -- 172.21.15.32:0/128421764 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7ff580007ca0 con 0x7ff5901019f0 2024-05-03T18:44:47.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:47.855+0000 7ff58cff9700 1 --2- 172.21.15.32:0/128421764 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7ff5780b2880 0x7ff5780b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:47.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:47.855+0000 7ff58effd700 1 --2- 172.21.15.32:0/128421764 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7ff5780b2880 0x7ff5780b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:44:47.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:47.855+0000 7ff58cff9700 1 -- 172.21.15.32:0/128421764 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7497+0+0 (secure 0 0 0) 0x7ff5800e1e50 con 0x7ff5901019f0 2024-05-03T18:44:47.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:47.855+0000 7ff58effd700 1 --2- 172.21.15.32:0/128421764 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7ff5780b2880 0x7ff5780b4d40 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff58400b580 tx=0x7ff5840096d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:44:47.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:47.859+0000 7ff58cff9700 1 -- 172.21.15.32:0/128421764 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7ff580014030 con 0x7ff5901019f0 2024-05-03T18:44:48.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:48.072+0000 7ff595d3b700 1 -- 172.21.15.32:0/128421764 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7ff5901a8b70 con 0x7ff5901019f0 2024-05-03T18:44:48.406 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:48.405+0000 7ff58cff9700 1 -- 172.21.15.32:0/128421764 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v75) ==== 170+0+0 (secure 0 0 0) 0x7ff5800a9ea0 con 0x7ff5901019f0 2024-05-03T18:44:48.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:48.408+0000 7ff595d3b700 1 -- 172.21.15.32:0/128421764 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7ff5780b2880 msgr2=0x7ff5780b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:44:48.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:48.408+0000 7ff595d3b700 1 --2- 172.21.15.32:0/128421764 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7ff5780b2880 0x7ff5780b4d40 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff58400b580 tx=0x7ff5840096d0 comp rx=0 tx=0).stop 2024-05-03T18:44:48.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:48.409+0000 7ff595d3b700 1 -- 172.21.15.32:0/128421764 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff5901019f0 msgr2=0x7ff59019bdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:44:48.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:48.409+0000 7ff595d3b700 1 --2- 172.21.15.32:0/128421764 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff5901019f0 0x7ff59019bdc0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ff58000c920 tx=0x7ff58000cc30 comp rx=0 tx=0).stop 2024-05-03T18:44:48.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:48.409+0000 7ff595d3b700 1 -- 172.21.15.32:0/128421764 shutdown_connections 2024-05-03T18:44:48.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:48.409+0000 7ff595d3b700 1 --2- 172.21.15.32:0/128421764 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7ff5780b2880 0x7ff5780b4d40 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:48.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:48.409+0000 7ff595d3b700 1 --2- 172.21.15.32:0/128421764 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff590109570 0x7ff5901a0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:48.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:48.409+0000 7ff595d3b700 1 --2- 172.21.15.32:0/128421764 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff5901019f0 0x7ff59019bdc0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:48.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:48.409+0000 7ff595d3b700 1 --2- 172.21.15.32:0/128421764 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff5901023b0 0x7ff59019c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:48.410 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:48.409+0000 7ff595d3b700 1 -- 172.21.15.32:0/128421764 >> 172.21.15.32:0/128421764 conn(0x7ff5900789a0 msgr2=0x7ff5900ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:44:48.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:48.409+0000 7ff595d3b700 1 -- 172.21.15.32:0/128421764 shutdown_connections 2024-05-03T18:44:48.411 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:48.410+0000 7ff595d3b700 1 -- 172.21.15.32:0/128421764 wait complete. 2024-05-03T18:44:48.415 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_data' created 2024-05-03T18:44:48.704 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:48 smithi045 ceph-mon[33299]: osdmap e74: 12 total, 12 up, 12 in 2024-05-03T18:44:48.704 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:48 smithi045 ceph-mon[33299]: from='client.? 172.21.15.32:0/128421764' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-03T18:44:48.714 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:48 smithi032 ceph-mon[29763]: osdmap e74: 12 total, 12 up, 12 in 2024-05-03T18:44:48.714 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:48 smithi032 ceph-mon[29763]: from='client.? 172.21.15.32:0/128421764' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-03T18:44:48.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:48 smithi113 ceph-mon[33540]: osdmap e74: 12 total, 12 up, 12 in 2024-05-03T18:44:48.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:48 smithi113 ceph-mon[33540]: from='client.? 172.21.15.32:0/128421764' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-03T18:44:49.047 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid f500455e-097b-11ef-bc97-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-03T18:44:49.298 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:44:49.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:49 smithi045 ceph-mon[33299]: pgmap v275: 33 pgs: 7 active+clean, 11 creating+peering, 15 unknown; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:44:49.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:49 smithi045 ceph-mon[33299]: from='client.? 172.21.15.32:0/128421764' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-03T18:44:49.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:49 smithi045 ceph-mon[33299]: osdmap e75: 12 total, 12 up, 12 in 2024-05-03T18:44:49.758 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:49 smithi032 ceph-mon[29763]: pgmap v275: 33 pgs: 7 active+clean, 11 creating+peering, 15 unknown; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:44:49.758 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:49 smithi032 ceph-mon[29763]: from='client.? 172.21.15.32:0/128421764' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-03T18:44:49.758 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:49 smithi032 ceph-mon[29763]: osdmap e75: 12 total, 12 up, 12 in 2024-05-03T18:44:49.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:49 smithi113 ceph-mon[33540]: pgmap v275: 33 pgs: 7 active+clean, 11 creating+peering, 15 unknown; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:44:49.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:49 smithi113 ceph-mon[33540]: from='client.? 172.21.15.32:0/128421764' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-03T18:44:49.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:49 smithi113 ceph-mon[33540]: osdmap e75: 12 total, 12 up, 12 in 2024-05-03T18:44:50.680 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:50 smithi032 ceph-mon[29763]: osdmap e76: 12 total, 12 up, 12 in 2024-05-03T18:44:50.747 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:50 smithi045 ceph-mon[33299]: osdmap e76: 12 total, 12 up, 12 in 2024-05-03T18:44:50.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:50 smithi113 ceph-mon[33540]: osdmap e76: 12 total, 12 up, 12 in 2024-05-03T18:44:50.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:50.883+0000 7fcc61bc9700 1 -- 172.21.15.32:0/1700892888 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc5c108410 msgr2=0x7fcc5c1087f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:44:50.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:50.883+0000 7fcc61bc9700 1 --2- 172.21.15.32:0/1700892888 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc5c108410 0x7fcc5c1087f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fcc4c009990 tx=0x7fcc4c009ca0 comp rx=0 tx=0).stop 2024-05-03T18:44:50.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:50.883+0000 7fcc61bc9700 1 -- 172.21.15.32:0/1700892888 shutdown_connections 2024-05-03T18:44:50.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:50.883+0000 7fcc61bc9700 1 --2- 172.21.15.32:0/1700892888 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcc5c102d60 0x7fcc5c10f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:50.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:50.883+0000 7fcc61bc9700 1 --2- 172.21.15.32:0/1700892888 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc5c108410 0x7fcc5c1087f0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:50.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:50.883+0000 7fcc61bc9700 1 --2- 172.21.15.32:0/1700892888 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcc5c1023a0 0x7fcc5c102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:50.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:50.883+0000 7fcc61bc9700 1 -- 172.21.15.32:0/1700892888 >> 172.21.15.32:0/1700892888 conn(0x7fcc5c079350 msgr2=0x7fcc5c079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:44:50.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:50.883+0000 7fcc61bc9700 1 -- 172.21.15.32:0/1700892888 shutdown_connections 2024-05-03T18:44:50.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:50.883+0000 7fcc61bc9700 1 -- 172.21.15.32:0/1700892888 wait complete. 2024-05-03T18:44:50.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:50.884+0000 7fcc61bc9700 1 Processor -- start 2024-05-03T18:44:50.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:50.884+0000 7fcc61bc9700 1 -- start start 2024-05-03T18:44:50.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:50.884+0000 7fcc61bc9700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcc5c1023a0 0x7fcc5c1043f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:50.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:50.884+0000 7fcc61bc9700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcc5c102d60 0x7fcc5c104930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:50.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:50.884+0000 7fcc61bc9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc5c108410 0x7fcc5c104e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:50.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:50.884+0000 7fcc61bc9700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fcc5c1125b0 con 0x7fcc5c108410 2024-05-03T18:44:50.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:50.884+0000 7fcc61bc9700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fcc5c112430 con 0x7fcc5c102d60 2024-05-03T18:44:50.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:50.884+0000 7fcc61bc9700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fcc5c112730 con 0x7fcc5c1023a0 2024-05-03T18:44:50.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:50.884+0000 7fcc5bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc5c108410 0x7fcc5c104e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:44:50.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:50.884+0000 7fcc5bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc5c108410 0x7fcc5c104e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58584/0 (socket says 172.21.15.32:58584) 2024-05-03T18:44:50.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:50.884+0000 7fcc5bfff700 1 -- 172.21.15.32:0/1320755762 learned_addr learned my addr 172.21.15.32:0/1320755762 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:44:50.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:50.884+0000 7fcc5b7fe700 1 --2- 172.21.15.32:0/1320755762 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcc5c1023a0 0x7fcc5c1043f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:44:50.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:50.884+0000 7fcc5affd700 1 --2- 172.21.15.32:0/1320755762 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcc5c102d60 0x7fcc5c104930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:44:50.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:50.885+0000 7fcc5bfff700 1 -- 172.21.15.32:0/1320755762 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcc5c1023a0 msgr2=0x7fcc5c1043f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:44:50.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:50.885+0000 7fcc5bfff700 1 --2- 172.21.15.32:0/1320755762 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcc5c1023a0 0x7fcc5c1043f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:50.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:50.885+0000 7fcc5bfff700 1 -- 172.21.15.32:0/1320755762 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcc5c102d60 msgr2=0x7fcc5c104930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:44:50.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:50.885+0000 7fcc5bfff700 1 --2- 172.21.15.32:0/1320755762 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcc5c102d60 0x7fcc5c104930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:50.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:50.885+0000 7fcc5bfff700 1 -- 172.21.15.32:0/1320755762 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc5c1a8140 con 0x7fcc5c108410 2024-05-03T18:44:50.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:50.885+0000 7fcc5bfff700 1 --2- 172.21.15.32:0/1320755762 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc5c108410 0x7fcc5c104e70 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fcc54007d70 tx=0x7fcc5400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:44:50.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:50.885+0000 7fcc58ff9700 1 -- 172.21.15.32:0/1320755762 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc54016020 con 0x7fcc5c108410 2024-05-03T18:44:50.888 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:50.885+0000 7fcc58ff9700 1 -- 172.21.15.32:0/1320755762 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc5400ad40 con 0x7fcc5c108410 2024-05-03T18:44:50.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:50.885+0000 7fcc61bc9700 1 -- 172.21.15.32:0/1320755762 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc5c1a8430 con 0x7fcc5c108410 2024-05-03T18:44:50.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:50.885+0000 7fcc58ff9700 1 -- 172.21.15.32:0/1320755762 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc540115e0 con 0x7fcc5c108410 2024-05-03T18:44:50.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:50.885+0000 7fcc61bc9700 1 -- 172.21.15.32:0/1320755762 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc5c1a8920 con 0x7fcc5c108410 2024-05-03T18:44:50.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:50.886+0000 7fcc61bc9700 1 -- 172.21.15.32:0/1320755762 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc5c06a190 con 0x7fcc5c108410 2024-05-03T18:44:50.889 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:50.888+0000 7fcc58ff9700 1 -- 172.21.15.32:0/1320755762 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fcc54019020 con 0x7fcc5c108410 2024-05-03T18:44:50.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:50.889+0000 7fcc58ff9700 1 --2- 172.21.15.32:0/1320755762 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fcc440b2880 0x7fcc440b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:50.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:50.889+0000 7fcc5b7fe700 1 --2- 172.21.15.32:0/1320755762 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fcc440b2880 0x7fcc440b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:44:50.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:50.889+0000 7fcc58ff9700 1 -- 172.21.15.32:0/1320755762 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7879+0+0 (secure 0 0 0) 0x7fcc540e25f0 con 0x7fcc5c108410 2024-05-03T18:44:50.890 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:50.889+0000 7fcc5b7fe700 1 --2- 172.21.15.32:0/1320755762 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fcc440b2880 0x7fcc440b4d40 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fcc4c00b5c0 tx=0x7fcc4c005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:44:50.893 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:50.893+0000 7fcc58ff9700 1 -- 172.21.15.32:0/1320755762 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fcc54020020 con 0x7fcc5c108410 2024-05-03T18:44:51.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:51.153+0000 7fcc61bc9700 1 -- 172.21.15.32:0/1320755762 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fcc5c1a8be0 con 0x7fcc5c108410 2024-05-03T18:44:51.419 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:51.418+0000 7fcc58ff9700 1 -- 172.21.15.32:0/1320755762 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7fcc540aa420 con 0x7fcc5c108410 2024-05-03T18:44:51.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:51.421+0000 7fcc61bc9700 1 -- 172.21.15.32:0/1320755762 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fcc440b2880 msgr2=0x7fcc440b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:44:51.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:51.421+0000 7fcc61bc9700 1 --2- 172.21.15.32:0/1320755762 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fcc440b2880 0x7fcc440b4d40 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fcc4c00b5c0 tx=0x7fcc4c005960 comp rx=0 tx=0).stop 2024-05-03T18:44:51.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:51.421+0000 7fcc61bc9700 1 -- 172.21.15.32:0/1320755762 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc5c108410 msgr2=0x7fcc5c104e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:44:51.422 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:51.421+0000 7fcc61bc9700 1 --2- 172.21.15.32:0/1320755762 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc5c108410 0x7fcc5c104e70 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fcc54007d70 tx=0x7fcc5400a340 comp rx=0 tx=0).stop 2024-05-03T18:44:51.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:51.421+0000 7fcc61bc9700 1 -- 172.21.15.32:0/1320755762 shutdown_connections 2024-05-03T18:44:51.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:51.421+0000 7fcc61bc9700 1 --2- 172.21.15.32:0/1320755762 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fcc440b2880 0x7fcc440b4d40 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:51.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:51.421+0000 7fcc61bc9700 1 --2- 172.21.15.32:0/1320755762 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc5c108410 0x7fcc5c104e70 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:51.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:51.421+0000 7fcc61bc9700 1 --2- 172.21.15.32:0/1320755762 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcc5c1023a0 0x7fcc5c1043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:51.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:51.421+0000 7fcc61bc9700 1 --2- 172.21.15.32:0/1320755762 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcc5c102d60 0x7fcc5c104930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:51.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:51.421+0000 7fcc61bc9700 1 -- 172.21.15.32:0/1320755762 >> 172.21.15.32:0/1320755762 conn(0x7fcc5c079350 msgr2=0x7fcc5c106770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:44:51.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:51.422+0000 7fcc61bc9700 1 -- 172.21.15.32:0/1320755762 shutdown_connections 2024-05-03T18:44:51.423 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:51.422+0000 7fcc61bc9700 1 -- 172.21.15.32:0/1320755762 wait complete. 2024-05-03T18:44:51.424 INFO:teuthology.orchestra.run.smithi032.stderr:new fs with metadata pool 2 and data pool 3 2024-05-03T18:44:51.727 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:51 smithi032 ceph-mon[29763]: pgmap v278: 97 pgs: 29 active+clean, 23 creating+peering, 45 unknown; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:44:51.727 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:51 smithi032 ceph-mon[29763]: osdmap e77: 12 total, 12 up, 12 in 2024-05-03T18:44:51.727 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:51 smithi032 ceph-mon[29763]: from='client.? 172.21.15.32:0/1320755762' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-03T18:44:51.727 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:51 smithi032 ceph-mon[29763]: daemon mds.i assigned to filesystem cephfs as rank 0 2024-05-03T18:44:51.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:51 smithi045 ceph-mon[33299]: pgmap v278: 97 pgs: 29 active+clean, 23 creating+peering, 45 unknown; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:44:51.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:51 smithi045 ceph-mon[33299]: osdmap e77: 12 total, 12 up, 12 in 2024-05-03T18:44:51.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:51 smithi045 ceph-mon[33299]: from='client.? 172.21.15.32:0/1320755762' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-03T18:44:51.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:51 smithi045 ceph-mon[33299]: daemon mds.i assigned to filesystem cephfs as rank 0 2024-05-03T18:44:51.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:51 smithi113 ceph-mon[33540]: pgmap v278: 97 pgs: 29 active+clean, 23 creating+peering, 45 unknown; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:44:51.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:51 smithi113 ceph-mon[33540]: osdmap e77: 12 total, 12 up, 12 in 2024-05-03T18:44:51.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:51 smithi113 ceph-mon[33540]: from='client.? 172.21.15.32:0/1320755762' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-03T18:44:51.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:51 smithi113 ceph-mon[33540]: daemon mds.i assigned to filesystem cephfs as rank 0 2024-05-03T18:44:52.021 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid f500455e-097b-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-03T18:44:52.274 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:44:52.656 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:52 smithi032 ceph-mon[29763]: from='client.? 172.21.15.32:0/1320755762' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-03T18:44:52.656 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:52 smithi032 ceph-mon[29763]: osdmap e78: 12 total, 12 up, 12 in 2024-05-03T18:44:52.656 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:52 smithi032 ceph-mon[29763]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2024-05-03T18:44:52.656 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:52 smithi032 ceph-mon[29763]: daemon mds.i is now active in filesystem cephfs as rank 0 2024-05-03T18:44:52.747 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:52 smithi045 ceph-mon[33299]: from='client.? 172.21.15.32:0/1320755762' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-03T18:44:52.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:52 smithi045 ceph-mon[33299]: osdmap e78: 12 total, 12 up, 12 in 2024-05-03T18:44:52.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:52 smithi045 ceph-mon[33299]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2024-05-03T18:44:52.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:52 smithi045 ceph-mon[33299]: daemon mds.i is now active in filesystem cephfs as rank 0 2024-05-03T18:44:52.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:52 smithi113 ceph-mon[33540]: from='client.? 172.21.15.32:0/1320755762' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-03T18:44:52.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:52 smithi113 ceph-mon[33540]: osdmap e78: 12 total, 12 up, 12 in 2024-05-03T18:44:52.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:52 smithi113 ceph-mon[33540]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2024-05-03T18:44:52.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:52 smithi113 ceph-mon[33540]: daemon mds.i is now active in filesystem cephfs as rank 0 2024-05-03T18:44:53.467 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:53 smithi032 ceph-mon[29763]: pgmap v281: 97 pgs: 79 active+clean, 12 creating+peering, 6 unknown; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:44:53.467 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:53 smithi032 ceph-mon[29763]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-05-03T18:44:53.467 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:53 smithi032 ceph-mon[29763]: Cluster is now healthy 2024-05-03T18:44:53.467 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:53 smithi032 ceph-mon[29763]: mds.? [v2:172.21.15.113:6838/942050693,v1:172.21.15.113:6839/942050693] up:active 2024-05-03T18:44:53.467 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:53 smithi032 ceph-mon[29763]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-03T18:44:53.467 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:53 smithi032 ceph-mon[29763]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-03T18:44:53.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:53.682+0000 7ff1cff2b700 1 -- 172.21.15.32:0/2315662318 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff1c8102fd0 msgr2=0x7ff1c8107020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:44:53.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:53.682+0000 7ff1cff2b700 1 --2- 172.21.15.32:0/2315662318 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff1c8102fd0 0x7ff1c8107020 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ff1b80099c0 tx=0x7ff1b8009cd0 comp rx=0 tx=0).stop 2024-05-03T18:44:53.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:53.682+0000 7ff1cff2b700 1 -- 172.21.15.32:0/2315662318 shutdown_connections 2024-05-03T18:44:53.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:53.682+0000 7ff1cff2b700 1 --2- 172.21.15.32:0/2315662318 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff1c8107700 0x7ff1c810f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:53.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:53.682+0000 7ff1cff2b700 1 --2- 172.21.15.32:0/2315662318 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1c8102620 0x7ff1c8102a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:53.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:53.682+0000 7ff1cff2b700 1 --2- 172.21.15.32:0/2315662318 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff1c8102fd0 0x7ff1c8107020 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:53.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:53.682+0000 7ff1cff2b700 1 -- 172.21.15.32:0/2315662318 >> 172.21.15.32:0/2315662318 conn(0x7ff1c8079440 msgr2=0x7ff1c8079850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:44:53.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:53.682+0000 7ff1cff2b700 1 -- 172.21.15.32:0/2315662318 shutdown_connections 2024-05-03T18:44:53.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:53.682+0000 7ff1cff2b700 1 -- 172.21.15.32:0/2315662318 wait complete. 2024-05-03T18:44:53.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:53.682+0000 7ff1cff2b700 1 Processor -- start 2024-05-03T18:44:53.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:53.683+0000 7ff1cff2b700 1 -- start start 2024-05-03T18:44:53.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:53.683+0000 7ff1cff2b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff1c8102620 0x7ff1c819e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:53.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:53.683+0000 7ff1cff2b700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff1c8102fd0 0x7ff1c819ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:53.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:53.683+0000 7ff1cff2b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1c8107700 0x7ff1c8198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:53.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:53.683+0000 7ff1cff2b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff1c8112570 con 0x7ff1c8102620 2024-05-03T18:44:53.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:53.683+0000 7ff1cff2b700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff1c81123f0 con 0x7ff1c8107700 2024-05-03T18:44:53.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:53.683+0000 7ff1cff2b700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7ff1c81126f0 con 0x7ff1c8102fd0 2024-05-03T18:44:53.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:53.683+0000 7ff1cdcc7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff1c8102620 0x7ff1c819e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:44:53.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:53.683+0000 7ff1cdcc7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff1c8102620 0x7ff1c819e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45694/0 (socket says 172.21.15.32:45694) 2024-05-03T18:44:53.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:53.683+0000 7ff1cdcc7700 1 -- 172.21.15.32:0/171542042 learned_addr learned my addr 172.21.15.32:0/171542042 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:44:53.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:53.683+0000 7ff1ce4c8700 1 --2- 172.21.15.32:0/171542042 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1c8107700 0x7ff1c8198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:44:53.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:53.683+0000 7ff1cd4c6700 1 --2- 172.21.15.32:0/171542042 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff1c8102fd0 0x7ff1c819ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:44:53.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:53.683+0000 7ff1cdcc7700 1 -- 172.21.15.32:0/171542042 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff1c8102fd0 msgr2=0x7ff1c819ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:44:53.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:53.683+0000 7ff1cdcc7700 1 --2- 172.21.15.32:0/171542042 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff1c8102fd0 0x7ff1c819ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:53.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:53.683+0000 7ff1cdcc7700 1 -- 172.21.15.32:0/171542042 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1c8107700 msgr2=0x7ff1c8198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:44:53.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:53.683+0000 7ff1cdcc7700 1 --2- 172.21.15.32:0/171542042 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1c8107700 0x7ff1c8198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:53.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:53.683+0000 7ff1cdcc7700 1 -- 172.21.15.32:0/171542042 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1c8199070 con 0x7ff1c8102620 2024-05-03T18:44:53.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:53.683+0000 7ff1ce4c8700 1 --2- 172.21.15.32:0/171542042 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1c8107700 0x7ff1c8198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T18:44:53.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:53.683+0000 7ff1cd4c6700 1 --2- 172.21.15.32:0/171542042 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff1c8102fd0 0x7ff1c819ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:44:53.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:53.683+0000 7ff1cdcc7700 1 --2- 172.21.15.32:0/171542042 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff1c8102620 0x7ff1c819e700 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ff1c400d950 tx=0x7ff1c400dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:44:53.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:53.684+0000 7ff1beffd700 1 -- 172.21.15.32:0/171542042 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1c4016070 con 0x7ff1c8102620 2024-05-03T18:44:53.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:53.684+0000 7ff1cff2b700 1 -- 172.21.15.32:0/171542042 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1c8199360 con 0x7ff1c8102620 2024-05-03T18:44:53.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:53.684+0000 7ff1beffd700 1 -- 172.21.15.32:0/171542042 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1c4010460 con 0x7ff1c8102620 2024-05-03T18:44:53.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:53.684+0000 7ff1beffd700 1 -- 172.21.15.32:0/171542042 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1c400f620 con 0x7ff1c8102620 2024-05-03T18:44:53.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:53.684+0000 7ff1cff2b700 1 -- 172.21.15.32:0/171542042 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1c81a55d0 con 0x7ff1c8102620 2024-05-03T18:44:53.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:53.684+0000 7ff1cff2b700 1 -- 172.21.15.32:0/171542042 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1c806a190 con 0x7ff1c8102620 2024-05-03T18:44:53.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:53.686+0000 7ff1beffd700 1 -- 172.21.15.32:0/171542042 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7ff1c4019070 con 0x7ff1c8102620 2024-05-03T18:44:53.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:53.687+0000 7ff1beffd700 1 --2- 172.21.15.32:0/171542042 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7ff1b40b2880 0x7ff1b40b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:53.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:53.687+0000 7ff1cd4c6700 1 --2- 172.21.15.32:0/171542042 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7ff1b40b2880 0x7ff1b40b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:44:53.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:53.687+0000 7ff1beffd700 1 -- 172.21.15.32:0/171542042 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7979+0+0 (secure 0 0 0) 0x7ff1c40e1d30 con 0x7ff1c8102620 2024-05-03T18:44:53.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:53.687+0000 7ff1cd4c6700 1 --2- 172.21.15.32:0/171542042 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7ff1b40b2880 0x7ff1b40b4d40 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ff1b8006010 tx=0x7ff1b8005e80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:44:53.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:53.690+0000 7ff1beffd700 1 -- 172.21.15.32:0/171542042 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7ff1c40a9ba0 con 0x7ff1c8102620 2024-05-03T18:44:53.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:53 smithi045 ceph-mon[33299]: pgmap v281: 97 pgs: 79 active+clean, 12 creating+peering, 6 unknown; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:44:53.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:53 smithi045 ceph-mon[33299]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-05-03T18:44:53.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:53 smithi045 ceph-mon[33299]: Cluster is now healthy 2024-05-03T18:44:53.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:53 smithi045 ceph-mon[33299]: mds.? [v2:172.21.15.113:6838/942050693,v1:172.21.15.113:6839/942050693] up:active 2024-05-03T18:44:53.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:53 smithi045 ceph-mon[33299]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-03T18:44:53.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:53 smithi045 ceph-mon[33299]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-03T18:44:53.801 INFO:journalctl@ceph.mds.i.smithi113.stdout:May 03 18:44:53 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-mds-i[65223]: 2024-05-03T18:44:53.461+0000 7f0d711c2700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-03T18:44:53.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:53 smithi113 ceph-mon[33540]: pgmap v281: 97 pgs: 79 active+clean, 12 creating+peering, 6 unknown; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T18:44:53.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:53 smithi113 ceph-mon[33540]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-05-03T18:44:53.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:53 smithi113 ceph-mon[33540]: Cluster is now healthy 2024-05-03T18:44:53.802 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:53 smithi113 ceph-mon[33540]: mds.? [v2:172.21.15.113:6838/942050693,v1:172.21.15.113:6839/942050693] up:active 2024-05-03T18:44:53.802 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:53 smithi113 ceph-mon[33540]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-03T18:44:53.802 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:53 smithi113 ceph-mon[33540]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-03T18:44:53.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:53.909+0000 7ff1cff2b700 1 -- 172.21.15.32:0/171542042 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff1c8199df0 con 0x7ff1c8102620 2024-05-03T18:44:53.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:53.911+0000 7ff1beffd700 1 -- 172.21.15.32:0/171542042 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) ==== 74+0+22148 (secure 0 0 0) 0x7ff1c40a94a0 con 0x7ff1c8102620 2024-05-03T18:44:53.912 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-03T18:44:53.912 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":78,"fsid":"f500455e-097b-11ef-bc97-c7b262605968","created":"2024-05-03T18:36:43.750772+0000","modified":"2024-05-03T18:44:51.155086+0000","last_up_change":"2024-05-03T18:43:50.933818+0000","last_in_change":"2024-05-03T18:43:31.804386+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-03T18:40:07.981853+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-03T18:44:44.607622+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-03T18:44:48.077391+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0430c3c0-2c50-471c-9c74-2869a0627c74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":262324956},{"type":"v1","addr":"172.21.15.32:6803","nonce":262324956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":262324956},{"type":"v1","addr":"172.21.15.32:6805","nonce":262324956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":262324956},{"type":"v1","addr":"172.21.15.32:6809","nonce":262324956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":262324956},{"type":"v1","addr":"172.21.15.32:6807","nonce":262324956}]},"public_addr":"172.21.15.32:6803/262324956","cluster_addr":"172.21.15.32:6805/262324956","heartbeat_back_addr":"172.21.15.32:6809/262324956","heartbeat_front_addr":"172.21.15.32:6807/262324956","state":["exists","up"]},{"osd":1,"uuid":"ba8f0b67-1d7d-4d4c-8276-eff664d21f02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1209907302},{"type":"v1","addr":"172.21.15.45:6801","nonce":1209907302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1209907302},{"type":"v1","addr":"172.21.15.45:6803","nonce":1209907302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1209907302},{"type":"v1","addr":"172.21.15.45:6807","nonce":1209907302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1209907302},{"type":"v1","addr":"172.21.15.45:6805","nonce":1209907302}]},"public_addr":"172.21.15.45:6801/1209907302","cluster_addr":"172.21.15.45:6803/1209907302","heartbeat_back_addr":"172.21.15.45:6807/1209907302","heartbeat_front_addr":"172.21.15.45:6805/1209907302","state":["exists","up"]},{"osd":2,"uuid":"5624eabc-be47-4bb8-9d8e-b84734769c83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":2308128926},{"type":"v1","addr":"172.21.15.113:6801","nonce":2308128926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":2308128926},{"type":"v1","addr":"172.21.15.113:6803","nonce":2308128926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":2308128926},{"type":"v1","addr":"172.21.15.113:6807","nonce":2308128926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":2308128926},{"type":"v1","addr":"172.21.15.113:6805","nonce":2308128926}]},"public_addr":"172.21.15.113:6801/2308128926","cluster_addr":"172.21.15.113:6803/2308128926","heartbeat_back_addr":"172.21.15.113:6807/2308128926","heartbeat_front_addr":"172.21.15.113:6805/2308128926","state":["exists","up"]},{"osd":3,"uuid":"62c9c977-afe1-4112-beba-b25af7caa2f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":2606514063},{"type":"v1","addr":"172.21.15.32:6811","nonce":2606514063}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":2606514063},{"type":"v1","addr":"172.21.15.32:6813","nonce":2606514063}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":2606514063},{"type":"v1","addr":"172.21.15.32:6817","nonce":2606514063}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":2606514063},{"type":"v1","addr":"172.21.15.32:6815","nonce":2606514063}]},"public_addr":"172.21.15.32:6811/2606514063","cluster_addr":"172.21.15.32:6813/2606514063","heartbeat_back_addr":"172.21.15.32:6817/2606514063","heartbeat_front_addr":"172.21.15.32:6815/2606514063","state":["exists","up"]},{"osd":4,"uuid":"99ad449f-b0bc-4aa1-b0cf-4a7a458126e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":71932118},{"type":"v1","addr":"172.21.15.45:6809","nonce":71932118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":71932118},{"type":"v1","addr":"172.21.15.45:6811","nonce":71932118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":71932118},{"type":"v1","addr":"172.21.15.45:6815","nonce":71932118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":71932118},{"type":"v1","addr":"172.21.15.45:6813","nonce":71932118}]},"public_addr":"172.21.15.45:6809/71932118","cluster_addr":"172.21.15.45:6811/71932118","heartbeat_back_addr":"172.21.15.45:6815/71932118","heartbeat_front_addr":"172.21.15.45:6813/71932118","state":["exists","up"]},{"osd":5,"uuid":"a6688ade-47d7-4a52-807f-3c880a8ffcbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":4085018597},{"type":"v1","addr":"172.21.15.113:6809","nonce":4085018597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":4085018597},{"type":"v1","addr":"172.21.15.113:6811","nonce":4085018597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":4085018597},{"type":"v1","addr":"172.21.15.113:6815","nonce":4085018597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":4085018597},{"type":"v1","addr":"172.21.15.113:6813","nonce":4085018597}]},"public_addr":"172.21.15.113:6809/4085018597","cluster_addr":"172.21.15.113:6811/4085018597","heartbeat_back_addr":"172.21.15.113:6815/4085018597","heartbeat_front_addr":"172.21.15.113:6813/4085018597","state":["exists","up"]},{"osd":6,"uuid":"1d9ebda9-55c7-473a-9dc0-fc60845bbfe9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":791407796},{"type":"v1","addr":"172.21.15.32:6819","nonce":791407796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":791407796},{"type":"v1","addr":"172.21.15.32:6821","nonce":791407796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":791407796},{"type":"v1","addr":"172.21.15.32:6825","nonce":791407796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":791407796},{"type":"v1","addr":"172.21.15.32:6823","nonce":791407796}]},"public_addr":"172.21.15.32:6819/791407796","cluster_addr":"172.21.15.32:6821/791407796","heartbeat_back_addr":"172.21.15.32:6825/791407796","heartbeat_front_addr":"172.21.15.32:6823/791407796","state":["exists","up"]},{"osd":7,"uuid":"988cce65-23ed-4814-ac09-177eb2f89fe1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1032572438},{"type":"v1","addr":"172.21.15.45:6817","nonce":1032572438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1032572438},{"type":"v1","addr":"172.21.15.45:6819","nonce":1032572438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1032572438},{"type":"v1","addr":"172.21.15.45:6823","nonce":1032572438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1032572438},{"type":"v1","addr":"172.21.15.45:6821","nonce":1032572438}]},"public_addr":"172.21.15.45:6817/1032572438","cluster_addr":"172.21.15.45:6819/1032572438","heartbeat_back_addr":"172.21.15.45:6823/1032572438","heartbeat_front_addr":"172.21.15.45:6821/1032572438","state":["exists","up"]},{"osd":8,"uuid":"816e8ce8-5afa-4b52-89be-f7793ed12a0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3569181558},{"type":"v1","addr":"172.21.15.113:6817","nonce":3569181558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3569181558},{"type":"v1","addr":"172.21.15.113:6819","nonce":3569181558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3569181558},{"type":"v1","addr":"172.21.15.113:6823","nonce":3569181558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3569181558},{"type":"v1","addr":"172.21.15.113:6821","nonce":3569181558}]},"public_addr":"172.21.15.113:6817/3569181558","cluster_addr":"172.21.15.113:6819/3569181558","heartbeat_back_addr":"172.21.15.113:6823/3569181558","heartbeat_front_addr":"172.21.15.113:6821/3569181558","state":["exists","up"]},{"osd":9,"uuid":"1d35c435-36b8-40e8-a2d4-0efe904f1107","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":4290196345},{"type":"v1","addr":"172.21.15.32:6827","nonce":4290196345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":4290196345},{"type":"v1","addr":"172.21.15.32:6829","nonce":4290196345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":4290196345},{"type":"v1","addr":"172.21.15.32:6833","nonce":4290196345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":4290196345},{"type":"v1","addr":"172.21.15.32:6831","nonce":4290196345}]},"public_addr":"172.21.15.32:6827/4290196345","cluster_addr":"172.21.15.32:6829/4290196345","heartbeat_back_addr":"172.21.15.32:6833/4290196345","heartbeat_front_addr":"172.21.15.32:6831/4290196345","state":["exists","up"]},{"osd":10,"uuid":"9af7879e-aa8d-4ef8-b289-4b145b614820","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1817064361},{"type":"v1","addr":"172.21.15.45:6825","nonce":1817064361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1817064361},{"type":"v1","addr":"172.21.15.45:6827","nonce":1817064361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1817064361},{"type":"v1","addr":"172.21.15.45:6831","nonce":1817064361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1817064361},{"type":"v1","addr":"172.21.15.45:6829","nonce":1817064361}]},"public_addr":"172.21.15.45:6825/1817064361","cluster_addr":"172.21.15.45:6827/1817064361","heartbeat_back_addr":"172.21.15.45:6831/1817064361","heartbeat_front_addr":"172.21.15.45:6829/1817064361","state":["exists","up"]},{"osd":11,"uuid":"22981709-1f9d-46f6-892a-5678394a2e1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":703464306},{"type":"v1","addr":"172.21.15.113:6825","nonce":703464306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":703464306},{"type":"v1","addr":"172.21.15.113:6827","nonce":703464306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":703464306},{"type":"v1","addr":"172.21.15.113:6831","nonce":703464306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":703464306},{"type":"v1","addr":"172.21.15.113:6829","nonce":703464306}]},"public_addr":"172.21.15.113:6825/703464306","cluster_addr":"172.21.15.113:6827/703464306","heartbeat_back_addr":"172.21.15.113:6831/703464306","heartbeat_front_addr":"172.21.15.113:6829/703464306","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:39:18.455575+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:39:42.234466+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:40:05.843072+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:40:30.822533+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:40:54.952487+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:41:18.750184+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:41:43.403326+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:42:08.285730+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:42:32.861740+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:42:59.307277+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:43:24.509691+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:43:49.850077+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:0/743758274":"2024-05-04T18:37:45.879602+0000","172.21.15.32:6801/2019702098":"2024-05-04T18:37:45.879602+0000","172.21.15.32:6800/2019702098":"2024-05-04T18:37:45.879602+0000","172.21.15.32:0/3192385439":"2024-05-04T18:37:18.492319+0000","172.21.15.32:0/1450308494":"2024-05-04T18:37:18.492319+0000","172.21.15.32:0/3957699240":"2024-05-04T18:37:45.879602+0000","172.21.15.32:0/3310503393":"2024-05-04T18:37:18.492319+0000","172.21.15.32:6801/3945405684":"2024-05-04T18:37:18.492319+0000","172.21.15.32:6800/3945405684":"2024-05-04T18:37:18.492319+0000","172.21.15.32:0/2471164468":"2024-05-04T18:37:45.879602+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-03T18:44:53.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:53.913+0000 7ff1cff2b700 1 -- 172.21.15.32:0/171542042 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7ff1b40b2880 msgr2=0x7ff1b40b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:44:53.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:53.913+0000 7ff1cff2b700 1 --2- 172.21.15.32:0/171542042 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7ff1b40b2880 0x7ff1b40b4d40 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ff1b8006010 tx=0x7ff1b8005e80 comp rx=0 tx=0).stop 2024-05-03T18:44:53.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:53.913+0000 7ff1cff2b700 1 -- 172.21.15.32:0/171542042 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff1c8102620 msgr2=0x7ff1c819e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:44:53.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:53.913+0000 7ff1cff2b700 1 --2- 172.21.15.32:0/171542042 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff1c8102620 0x7ff1c819e700 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ff1c400d950 tx=0x7ff1c400dd10 comp rx=0 tx=0).stop 2024-05-03T18:44:53.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:53.913+0000 7ff1cff2b700 1 -- 172.21.15.32:0/171542042 shutdown_connections 2024-05-03T18:44:53.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:53.913+0000 7ff1cff2b700 1 --2- 172.21.15.32:0/171542042 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7ff1b40b2880 0x7ff1b40b4d40 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:53.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:53.913+0000 7ff1cff2b700 1 --2- 172.21.15.32:0/171542042 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1c8107700 0x7ff1c8198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:53.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:53.913+0000 7ff1cff2b700 1 --2- 172.21.15.32:0/171542042 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff1c8102620 0x7ff1c819e700 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:53.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:53.913+0000 7ff1cff2b700 1 --2- 172.21.15.32:0/171542042 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff1c8102fd0 0x7ff1c819ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:53.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:53.913+0000 7ff1cff2b700 1 -- 172.21.15.32:0/171542042 >> 172.21.15.32:0/171542042 conn(0x7ff1c8079440 msgr2=0x7ff1c81005c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:44:53.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:53.913+0000 7ff1cff2b700 1 -- 172.21.15.32:0/171542042 shutdown_connections 2024-05-03T18:44:53.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:53.913+0000 7ff1cff2b700 1 -- 172.21.15.32:0/171542042 wait complete. 2024-05-03T18:44:54.470 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:54 smithi032 ceph-mon[29763]: from='client.? 172.21.15.32:0/171542042' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T18:44:54.472 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid f500455e-097b-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-03T18:44:54.724 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:44:54.747 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:54 smithi045 ceph-mon[33299]: from='client.? 172.21.15.32:0/171542042' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T18:44:54.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:54 smithi113 ceph-mon[33540]: from='client.? 172.21.15.32:0/171542042' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T18:44:55.702 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:55 smithi113 ceph-mon[33540]: pgmap v282: 97 pgs: 97 active+clean; 578 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 2 op/s 2024-05-03T18:44:55.747 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:55 smithi045 ceph-mon[33299]: pgmap v282: 97 pgs: 97 active+clean; 578 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 2 op/s 2024-05-03T18:44:55.754 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:55 smithi032 ceph-mon[29763]: pgmap v282: 97 pgs: 97 active+clean; 578 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 2 op/s 2024-05-03T18:44:56.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:56.280+0000 7fbfa0a51700 1 -- 172.21.15.32:0/2771178647 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf9c104100 msgr2=0x7fbf9c108120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:44:56.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:56.280+0000 7fbfa0a51700 1 --2- 172.21.15.32:0/2771178647 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf9c104100 0x7fbf9c108120 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fbf900099c0 tx=0x7fbf90009cd0 comp rx=0 tx=0).stop 2024-05-03T18:44:56.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:56.280+0000 7fbfa0a51700 1 -- 172.21.15.32:0/2771178647 shutdown_connections 2024-05-03T18:44:56.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:56.280+0000 7fbfa0a51700 1 --2- 172.21.15.32:0/2771178647 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbf9c108850 0x7fbf9c1103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:56.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:56.280+0000 7fbfa0a51700 1 --2- 172.21.15.32:0/2771178647 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf9c103750 0x7fbf9c103b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:56.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:56.280+0000 7fbfa0a51700 1 --2- 172.21.15.32:0/2771178647 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf9c104100 0x7fbf9c108120 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:56.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:56.280+0000 7fbfa0a51700 1 -- 172.21.15.32:0/2771178647 >> 172.21.15.32:0/2771178647 conn(0x7fbf9c07a600 msgr2=0x7fbf9c07aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:44:56.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:56.280+0000 7fbfa0a51700 1 -- 172.21.15.32:0/2771178647 shutdown_connections 2024-05-03T18:44:56.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:56.280+0000 7fbfa0a51700 1 -- 172.21.15.32:0/2771178647 wait complete. 2024-05-03T18:44:56.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:56.281+0000 7fbfa0a51700 1 Processor -- start 2024-05-03T18:44:56.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:56.281+0000 7fbfa0a51700 1 -- start start 2024-05-03T18:44:56.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:56.281+0000 7fbfa0a51700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf9c103750 0x7fbf9c19f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:56.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:56.281+0000 7fbfa0a51700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf9c104100 0x7fbf9c19fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:56.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:56.281+0000 7fbfa0a51700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbf9c108850 0x7fbf9c199970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:56.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:56.281+0000 7fbfa0a51700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fbf9c113690 con 0x7fbf9c104100 2024-05-03T18:44:56.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:56.281+0000 7fbfa0a51700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fbf9c113510 con 0x7fbf9c103750 2024-05-03T18:44:56.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:56.281+0000 7fbfa0a51700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fbf9c113810 con 0x7fbf9c108850 2024-05-03T18:44:56.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:56.281+0000 7fbf99d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf9c104100 0x7fbf9c19fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:44:56.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:56.281+0000 7fbf99d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf9c104100 0x7fbf9c19fcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45720/0 (socket says 172.21.15.32:45720) 2024-05-03T18:44:56.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:56.281+0000 7fbf99d9b700 1 -- 172.21.15.32:0/2490254858 learned_addr learned my addr 172.21.15.32:0/2490254858 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:44:56.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:56.281+0000 7fbf9a59c700 1 --2- 172.21.15.32:0/2490254858 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf9c103750 0x7fbf9c19f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:44:56.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:56.281+0000 7fbf99d9b700 1 -- 172.21.15.32:0/2490254858 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbf9c108850 msgr2=0x7fbf9c199970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:44:56.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:56.281+0000 7fbf99d9b700 1 --2- 172.21.15.32:0/2490254858 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbf9c108850 0x7fbf9c199970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:56.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:56.281+0000 7fbf99d9b700 1 -- 172.21.15.32:0/2490254858 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf9c103750 msgr2=0x7fbf9c19f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:44:56.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:56.281+0000 7fbf99d9b700 1 --2- 172.21.15.32:0/2490254858 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf9c103750 0x7fbf9c19f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:56.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:56.281+0000 7fbf99d9b700 1 -- 172.21.15.32:0/2490254858 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf9c19a0d0 con 0x7fbf9c104100 2024-05-03T18:44:56.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:56.282+0000 7fbf99d9b700 1 --2- 172.21.15.32:0/2490254858 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf9c104100 0x7fbf9c19fcf0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fbf9000b580 tx=0x7fbf900047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:44:56.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:56.282+0000 7fbf8b7fe700 1 -- 172.21.15.32:0/2490254858 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf90027070 con 0x7fbf9c104100 2024-05-03T18:44:56.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:56.282+0000 7fbfa0a51700 1 -- 172.21.15.32:0/2490254858 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf9c19a360 con 0x7fbf9c104100 2024-05-03T18:44:56.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:56.282+0000 7fbf8b7fe700 1 -- 172.21.15.32:0/2490254858 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf90004020 con 0x7fbf9c104100 2024-05-03T18:44:56.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:56.282+0000 7fbf8b7fe700 1 -- 172.21.15.32:0/2490254858 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf90022e30 con 0x7fbf9c104100 2024-05-03T18:44:56.287 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:56.282+0000 7fbfa0a51700 1 -- 172.21.15.32:0/2490254858 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf9c1a6680 con 0x7fbf9c104100 2024-05-03T18:44:56.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:56.283+0000 7fbfa0a51700 1 -- 172.21.15.32:0/2490254858 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf9c104be0 con 0x7fbf9c104100 2024-05-03T18:44:56.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:56.284+0000 7fbf8b7fe700 1 -- 172.21.15.32:0/2490254858 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fbf90022590 con 0x7fbf9c104100 2024-05-03T18:44:56.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:56.285+0000 7fbf8b7fe700 1 --2- 172.21.15.32:0/2490254858 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fbf840b2880 0x7fbf840b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:56.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:56.285+0000 7fbf9a59c700 1 --2- 172.21.15.32:0/2490254858 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fbf840b2880 0x7fbf840b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:44:56.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:56.285+0000 7fbf8b7fe700 1 -- 172.21.15.32:0/2490254858 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7979+0+0 (secure 0 0 0) 0x7fbf900e3300 con 0x7fbf9c104100 2024-05-03T18:44:56.288 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:56.285+0000 7fbf9a59c700 1 --2- 172.21.15.32:0/2490254858 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fbf840b2880 0x7fbf840b4d40 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fbf8c005950 tx=0x7fbf8c0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:44:56.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:56.288+0000 7fbf8b7fe700 1 -- 172.21.15.32:0/2490254858 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fbf90030070 con 0x7fbf9c104100 2024-05-03T18:44:56.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:56.501+0000 7fbfa0a51700 1 -- 172.21.15.32:0/2490254858 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbf9c02e510 con 0x7fbf9c104100 2024-05-03T18:44:56.504 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:56.503+0000 7fbf8b7fe700 1 -- 172.21.15.32:0/2490254858 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) ==== 74+0+22148 (secure 0 0 0) 0x7fbf90013070 con 0x7fbf9c104100 2024-05-03T18:44:56.505 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-03T18:44:56.505 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":78,"fsid":"f500455e-097b-11ef-bc97-c7b262605968","created":"2024-05-03T18:36:43.750772+0000","modified":"2024-05-03T18:44:51.155086+0000","last_up_change":"2024-05-03T18:43:50.933818+0000","last_in_change":"2024-05-03T18:43:31.804386+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-03T18:40:07.981853+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-03T18:44:44.607622+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-03T18:44:48.077391+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0430c3c0-2c50-471c-9c74-2869a0627c74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":262324956},{"type":"v1","addr":"172.21.15.32:6803","nonce":262324956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":262324956},{"type":"v1","addr":"172.21.15.32:6805","nonce":262324956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":262324956},{"type":"v1","addr":"172.21.15.32:6809","nonce":262324956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":262324956},{"type":"v1","addr":"172.21.15.32:6807","nonce":262324956}]},"public_addr":"172.21.15.32:6803/262324956","cluster_addr":"172.21.15.32:6805/262324956","heartbeat_back_addr":"172.21.15.32:6809/262324956","heartbeat_front_addr":"172.21.15.32:6807/262324956","state":["exists","up"]},{"osd":1,"uuid":"ba8f0b67-1d7d-4d4c-8276-eff664d21f02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1209907302},{"type":"v1","addr":"172.21.15.45:6801","nonce":1209907302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1209907302},{"type":"v1","addr":"172.21.15.45:6803","nonce":1209907302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1209907302},{"type":"v1","addr":"172.21.15.45:6807","nonce":1209907302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1209907302},{"type":"v1","addr":"172.21.15.45:6805","nonce":1209907302}]},"public_addr":"172.21.15.45:6801/1209907302","cluster_addr":"172.21.15.45:6803/1209907302","heartbeat_back_addr":"172.21.15.45:6807/1209907302","heartbeat_front_addr":"172.21.15.45:6805/1209907302","state":["exists","up"]},{"osd":2,"uuid":"5624eabc-be47-4bb8-9d8e-b84734769c83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":2308128926},{"type":"v1","addr":"172.21.15.113:6801","nonce":2308128926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":2308128926},{"type":"v1","addr":"172.21.15.113:6803","nonce":2308128926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":2308128926},{"type":"v1","addr":"172.21.15.113:6807","nonce":2308128926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":2308128926},{"type":"v1","addr":"172.21.15.113:6805","nonce":2308128926}]},"public_addr":"172.21.15.113:6801/2308128926","cluster_addr":"172.21.15.113:6803/2308128926","heartbeat_back_addr":"172.21.15.113:6807/2308128926","heartbeat_front_addr":"172.21.15.113:6805/2308128926","state":["exists","up"]},{"osd":3,"uuid":"62c9c977-afe1-4112-beba-b25af7caa2f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":2606514063},{"type":"v1","addr":"172.21.15.32:6811","nonce":2606514063}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":2606514063},{"type":"v1","addr":"172.21.15.32:6813","nonce":2606514063}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":2606514063},{"type":"v1","addr":"172.21.15.32:6817","nonce":2606514063}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":2606514063},{"type":"v1","addr":"172.21.15.32:6815","nonce":2606514063}]},"public_addr":"172.21.15.32:6811/2606514063","cluster_addr":"172.21.15.32:6813/2606514063","heartbeat_back_addr":"172.21.15.32:6817/2606514063","heartbeat_front_addr":"172.21.15.32:6815/2606514063","state":["exists","up"]},{"osd":4,"uuid":"99ad449f-b0bc-4aa1-b0cf-4a7a458126e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":71932118},{"type":"v1","addr":"172.21.15.45:6809","nonce":71932118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":71932118},{"type":"v1","addr":"172.21.15.45:6811","nonce":71932118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":71932118},{"type":"v1","addr":"172.21.15.45:6815","nonce":71932118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":71932118},{"type":"v1","addr":"172.21.15.45:6813","nonce":71932118}]},"public_addr":"172.21.15.45:6809/71932118","cluster_addr":"172.21.15.45:6811/71932118","heartbeat_back_addr":"172.21.15.45:6815/71932118","heartbeat_front_addr":"172.21.15.45:6813/71932118","state":["exists","up"]},{"osd":5,"uuid":"a6688ade-47d7-4a52-807f-3c880a8ffcbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":4085018597},{"type":"v1","addr":"172.21.15.113:6809","nonce":4085018597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":4085018597},{"type":"v1","addr":"172.21.15.113:6811","nonce":4085018597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":4085018597},{"type":"v1","addr":"172.21.15.113:6815","nonce":4085018597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":4085018597},{"type":"v1","addr":"172.21.15.113:6813","nonce":4085018597}]},"public_addr":"172.21.15.113:6809/4085018597","cluster_addr":"172.21.15.113:6811/4085018597","heartbeat_back_addr":"172.21.15.113:6815/4085018597","heartbeat_front_addr":"172.21.15.113:6813/4085018597","state":["exists","up"]},{"osd":6,"uuid":"1d9ebda9-55c7-473a-9dc0-fc60845bbfe9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":791407796},{"type":"v1","addr":"172.21.15.32:6819","nonce":791407796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":791407796},{"type":"v1","addr":"172.21.15.32:6821","nonce":791407796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":791407796},{"type":"v1","addr":"172.21.15.32:6825","nonce":791407796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":791407796},{"type":"v1","addr":"172.21.15.32:6823","nonce":791407796}]},"public_addr":"172.21.15.32:6819/791407796","cluster_addr":"172.21.15.32:6821/791407796","heartbeat_back_addr":"172.21.15.32:6825/791407796","heartbeat_front_addr":"172.21.15.32:6823/791407796","state":["exists","up"]},{"osd":7,"uuid":"988cce65-23ed-4814-ac09-177eb2f89fe1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1032572438},{"type":"v1","addr":"172.21.15.45:6817","nonce":1032572438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1032572438},{"type":"v1","addr":"172.21.15.45:6819","nonce":1032572438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1032572438},{"type":"v1","addr":"172.21.15.45:6823","nonce":1032572438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1032572438},{"type":"v1","addr":"172.21.15.45:6821","nonce":1032572438}]},"public_addr":"172.21.15.45:6817/1032572438","cluster_addr":"172.21.15.45:6819/1032572438","heartbeat_back_addr":"172.21.15.45:6823/1032572438","heartbeat_front_addr":"172.21.15.45:6821/1032572438","state":["exists","up"]},{"osd":8,"uuid":"816e8ce8-5afa-4b52-89be-f7793ed12a0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3569181558},{"type":"v1","addr":"172.21.15.113:6817","nonce":3569181558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3569181558},{"type":"v1","addr":"172.21.15.113:6819","nonce":3569181558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3569181558},{"type":"v1","addr":"172.21.15.113:6823","nonce":3569181558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3569181558},{"type":"v1","addr":"172.21.15.113:6821","nonce":3569181558}]},"public_addr":"172.21.15.113:6817/3569181558","cluster_addr":"172.21.15.113:6819/3569181558","heartbeat_back_addr":"172.21.15.113:6823/3569181558","heartbeat_front_addr":"172.21.15.113:6821/3569181558","state":["exists","up"]},{"osd":9,"uuid":"1d35c435-36b8-40e8-a2d4-0efe904f1107","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":4290196345},{"type":"v1","addr":"172.21.15.32:6827","nonce":4290196345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":4290196345},{"type":"v1","addr":"172.21.15.32:6829","nonce":4290196345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":4290196345},{"type":"v1","addr":"172.21.15.32:6833","nonce":4290196345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":4290196345},{"type":"v1","addr":"172.21.15.32:6831","nonce":4290196345}]},"public_addr":"172.21.15.32:6827/4290196345","cluster_addr":"172.21.15.32:6829/4290196345","heartbeat_back_addr":"172.21.15.32:6833/4290196345","heartbeat_front_addr":"172.21.15.32:6831/4290196345","state":["exists","up"]},{"osd":10,"uuid":"9af7879e-aa8d-4ef8-b289-4b145b614820","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1817064361},{"type":"v1","addr":"172.21.15.45:6825","nonce":1817064361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1817064361},{"type":"v1","addr":"172.21.15.45:6827","nonce":1817064361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1817064361},{"type":"v1","addr":"172.21.15.45:6831","nonce":1817064361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1817064361},{"type":"v1","addr":"172.21.15.45:6829","nonce":1817064361}]},"public_addr":"172.21.15.45:6825/1817064361","cluster_addr":"172.21.15.45:6827/1817064361","heartbeat_back_addr":"172.21.15.45:6831/1817064361","heartbeat_front_addr":"172.21.15.45:6829/1817064361","state":["exists","up"]},{"osd":11,"uuid":"22981709-1f9d-46f6-892a-5678394a2e1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":703464306},{"type":"v1","addr":"172.21.15.113:6825","nonce":703464306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":703464306},{"type":"v1","addr":"172.21.15.113:6827","nonce":703464306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":703464306},{"type":"v1","addr":"172.21.15.113:6831","nonce":703464306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":703464306},{"type":"v1","addr":"172.21.15.113:6829","nonce":703464306}]},"public_addr":"172.21.15.113:6825/703464306","cluster_addr":"172.21.15.113:6827/703464306","heartbeat_back_addr":"172.21.15.113:6831/703464306","heartbeat_front_addr":"172.21.15.113:6829/703464306","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:39:18.455575+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:39:42.234466+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:40:05.843072+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:40:30.822533+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:40:54.952487+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:41:18.750184+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:41:43.403326+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:42:08.285730+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:42:32.861740+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:42:59.307277+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:43:24.509691+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:43:49.850077+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:0/743758274":"2024-05-04T18:37:45.879602+0000","172.21.15.32:6801/2019702098":"2024-05-04T18:37:45.879602+0000","172.21.15.32:6800/2019702098":"2024-05-04T18:37:45.879602+0000","172.21.15.32:0/3192385439":"2024-05-04T18:37:18.492319+0000","172.21.15.32:0/1450308494":"2024-05-04T18:37:18.492319+0000","172.21.15.32:0/3957699240":"2024-05-04T18:37:45.879602+0000","172.21.15.32:0/3310503393":"2024-05-04T18:37:18.492319+0000","172.21.15.32:6801/3945405684":"2024-05-04T18:37:18.492319+0000","172.21.15.32:6800/3945405684":"2024-05-04T18:37:18.492319+0000","172.21.15.32:0/2471164468":"2024-05-04T18:37:45.879602+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-03T18:44:56.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:56.505+0000 7fbfa0a51700 1 -- 172.21.15.32:0/2490254858 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fbf840b2880 msgr2=0x7fbf840b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:44:56.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:56.505+0000 7fbfa0a51700 1 --2- 172.21.15.32:0/2490254858 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fbf840b2880 0x7fbf840b4d40 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fbf8c005950 tx=0x7fbf8c0058e0 comp rx=0 tx=0).stop 2024-05-03T18:44:56.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:56.505+0000 7fbfa0a51700 1 -- 172.21.15.32:0/2490254858 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf9c104100 msgr2=0x7fbf9c19fcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:44:56.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:56.505+0000 7fbfa0a51700 1 --2- 172.21.15.32:0/2490254858 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf9c104100 0x7fbf9c19fcf0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fbf9000b580 tx=0x7fbf900047b0 comp rx=0 tx=0).stop 2024-05-03T18:44:56.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:56.505+0000 7fbfa0a51700 1 -- 172.21.15.32:0/2490254858 shutdown_connections 2024-05-03T18:44:56.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:56.505+0000 7fbfa0a51700 1 --2- 172.21.15.32:0/2490254858 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fbf840b2880 0x7fbf840b4d40 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:56.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:56.505+0000 7fbfa0a51700 1 --2- 172.21.15.32:0/2490254858 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbf9c108850 0x7fbf9c199970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:56.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:56.505+0000 7fbfa0a51700 1 --2- 172.21.15.32:0/2490254858 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf9c103750 0x7fbf9c19f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:56.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:56.505+0000 7fbfa0a51700 1 --2- 172.21.15.32:0/2490254858 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf9c104100 0x7fbf9c19fcf0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:56.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:56.505+0000 7fbfa0a51700 1 -- 172.21.15.32:0/2490254858 >> 172.21.15.32:0/2490254858 conn(0x7fbf9c07a600 msgr2=0x7fbf9c101710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:44:56.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:56.505+0000 7fbfa0a51700 1 -- 172.21.15.32:0/2490254858 shutdown_connections 2024-05-03T18:44:56.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:56.505+0000 7fbfa0a51700 1 -- 172.21.15.32:0/2490254858 wait complete. 2024-05-03T18:44:57.165 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid f500455e-097b-11ef-bc97-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-05-03T18:44:57.420 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:44:57.747 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:57 smithi045 ceph-mon[33299]: pgmap v283: 97 pgs: 97 active+clean; 578 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 6 op/s 2024-05-03T18:44:57.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:57 smithi045 ceph-mon[33299]: from='client.? 172.21.15.32:0/2490254858' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T18:44:57.758 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:57 smithi032 ceph-mon[29763]: pgmap v283: 97 pgs: 97 active+clean; 578 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 6 op/s 2024-05-03T18:44:57.758 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:57 smithi032 ceph-mon[29763]: from='client.? 172.21.15.32:0/2490254858' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T18:44:57.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:57 smithi113 ceph-mon[33540]: pgmap v283: 97 pgs: 97 active+clean; 578 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 6 op/s 2024-05-03T18:44:57.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:57 smithi113 ceph-mon[33540]: from='client.? 172.21.15.32:0/2490254858' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T18:44:58.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:58.991+0000 7f00a5be1700 1 -- 172.21.15.32:0/1373152794 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f00a0107700 msgr2=0x7f00a010f2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:44:58.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:58.991+0000 7f00a5be1700 1 --2- 172.21.15.32:0/1373152794 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f00a0107700 0x7f00a010f2d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f009800b0d0 tx=0x7f009800b3e0 comp rx=0 tx=0).stop 2024-05-03T18:44:58.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:58.991+0000 7f00a5be1700 1 -- 172.21.15.32:0/1373152794 shutdown_connections 2024-05-03T18:44:58.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:58.991+0000 7f00a5be1700 1 --2- 172.21.15.32:0/1373152794 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f00a0107700 0x7f00a010f2d0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:58.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:58.991+0000 7f00a5be1700 1 --2- 172.21.15.32:0/1373152794 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f00a0102620 0x7f00a0102a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:58.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:58.991+0000 7f00a5be1700 1 --2- 172.21.15.32:0/1373152794 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f00a0102fd0 0x7f00a0107020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:58.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:58.991+0000 7f00a5be1700 1 -- 172.21.15.32:0/1373152794 >> 172.21.15.32:0/1373152794 conn(0x7f00a0079440 msgr2=0x7f00a0079850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:44:58.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:58.991+0000 7f00a5be1700 1 -- 172.21.15.32:0/1373152794 shutdown_connections 2024-05-03T18:44:58.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:58.991+0000 7f00a5be1700 1 -- 172.21.15.32:0/1373152794 wait complete. 2024-05-03T18:44:58.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:58.992+0000 7f00a5be1700 1 Processor -- start 2024-05-03T18:44:58.995 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:58.992+0000 7f00a5be1700 1 -- start start 2024-05-03T18:44:58.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:58.992+0000 7f00a5be1700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f00a0102620 0x7f00a019f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:58.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:58.993+0000 7f00a5be1700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f00a0102fd0 0x7f00a019fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:58.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:58.993+0000 7f00a5be1700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f00a0107700 0x7f00a0199910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:58.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:58.993+0000 7f00a5be1700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f00a01136f0 con 0x7f00a0102620 2024-05-03T18:44:58.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:58.993+0000 7f00a5be1700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f00a0113570 con 0x7f00a0107700 2024-05-03T18:44:58.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:58.993+0000 7f00a5be1700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f00a0113870 con 0x7f00a0102fd0 2024-05-03T18:44:58.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:58.993+0000 7f009f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f00a0102620 0x7f00a019f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:44:58.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:58.993+0000 7f009ffff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f00a0107700 0x7f00a0199910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:44:58.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:58.993+0000 7f009f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f00a0102620 0x7f00a019f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45738/0 (socket says 172.21.15.32:45738) 2024-05-03T18:44:58.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:58.993+0000 7f009f7fe700 1 -- 172.21.15.32:0/312330392 learned_addr learned my addr 172.21.15.32:0/312330392 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:44:58.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:58.993+0000 7f009effd700 1 --2- 172.21.15.32:0/312330392 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f00a0102fd0 0x7f00a019fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:44:58.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:58.993+0000 7f009ffff700 1 -- 172.21.15.32:0/312330392 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f00a0102fd0 msgr2=0x7f00a019fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:44:58.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:58.993+0000 7f009ffff700 1 --2- 172.21.15.32:0/312330392 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f00a0102fd0 0x7f00a019fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:58.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:58.993+0000 7f009ffff700 1 -- 172.21.15.32:0/312330392 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f00a0102620 msgr2=0x7f00a019f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:44:58.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:58.993+0000 7f009ffff700 1 --2- 172.21.15.32:0/312330392 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f00a0102620 0x7f00a019f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:58.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:58.993+0000 7f009ffff700 1 -- 172.21.15.32:0/312330392 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00a019a070 con 0x7f00a0107700 2024-05-03T18:44:58.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:58.993+0000 7f009f7fe700 1 --2- 172.21.15.32:0/312330392 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f00a0102620 0x7f00a019f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T18:44:58.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:58.993+0000 7f009ffff700 1 --2- 172.21.15.32:0/312330392 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f00a0107700 0x7f00a0199910 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0098009fd0 tx=0x7f0098007a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:44:58.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:58.993+0000 7f009cff9700 1 -- 172.21.15.32:0/312330392 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0098021070 con 0x7f00a0107700 2024-05-03T18:44:58.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:58.994+0000 7f00a5be1700 1 -- 172.21.15.32:0/312330392 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00a019a300 con 0x7f00a0107700 2024-05-03T18:44:58.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:58.994+0000 7f009cff9700 1 -- 172.21.15.32:0/312330392 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0098007e20 con 0x7f00a0107700 2024-05-03T18:44:58.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:58.994+0000 7f009cff9700 1 -- 172.21.15.32:0/312330392 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f009801b930 con 0x7f00a0107700 2024-05-03T18:44:58.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:58.994+0000 7f00a5be1700 1 -- 172.21.15.32:0/312330392 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00a01a6620 con 0x7f00a0107700 2024-05-03T18:44:58.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:58.995+0000 7f00a5be1700 1 -- 172.21.15.32:0/312330392 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00a006a190 con 0x7f00a0107700 2024-05-03T18:44:58.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:58.996+0000 7f009cff9700 1 -- 172.21.15.32:0/312330392 <== mon.2 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f0098024070 con 0x7f00a0107700 2024-05-03T18:44:58.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:58.997+0000 7f009cff9700 1 --2- 172.21.15.32:0/312330392 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f00880b2880 0x7f00880b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:44:58.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:58.997+0000 7f009f7fe700 1 --2- 172.21.15.32:0/312330392 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f00880b2880 0x7f00880b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:44:58.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:58.997+0000 7f009cff9700 1 -- 172.21.15.32:0/312330392 <== mon.2 v2:172.21.15.45:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7979+0+0 (secure 0 0 0) 0x7f009800f020 con 0x7f00a0107700 2024-05-03T18:44:58.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:58.997+0000 7f009f7fe700 1 --2- 172.21.15.32:0/312330392 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f00880b2880 0x7f00880b4d40 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f0090005950 tx=0x7f009000a3b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:44:59.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:59.001+0000 7f009cff9700 1 -- 172.21.15.32:0/312330392 <== mon.2 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f009802b020 con 0x7f00a0107700 2024-05-03T18:44:59.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:59.262+0000 7f00a5be1700 1 -- 172.21.15.32:0/312330392 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f00a002e510 con 0x7f00a0107700 2024-05-03T18:44:59.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:59.451+0000 7f009cff9700 1 -- 172.21.15.32:0/312330392 <== mon.2 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7f00980ab890 con 0x7f00a0107700 2024-05-03T18:44:59.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:59.453+0000 7f00a5be1700 1 -- 172.21.15.32:0/312330392 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f00880b2880 msgr2=0x7f00880b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:44:59.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:59.453+0000 7f00a5be1700 1 --2- 172.21.15.32:0/312330392 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f00880b2880 0x7f00880b4d40 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f0090005950 tx=0x7f009000a3b0 comp rx=0 tx=0).stop 2024-05-03T18:44:59.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:59.453+0000 7f00a5be1700 1 -- 172.21.15.32:0/312330392 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f00a0107700 msgr2=0x7f00a0199910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:44:59.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:59.453+0000 7f00a5be1700 1 --2- 172.21.15.32:0/312330392 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f00a0107700 0x7f00a0199910 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0098009fd0 tx=0x7f0098007a20 comp rx=0 tx=0).stop 2024-05-03T18:44:59.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:59.453+0000 7f00a5be1700 1 -- 172.21.15.32:0/312330392 shutdown_connections 2024-05-03T18:44:59.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:59.453+0000 7f00a5be1700 1 --2- 172.21.15.32:0/312330392 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f00880b2880 0x7f00880b4d40 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:59.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:59.453+0000 7f00a5be1700 1 --2- 172.21.15.32:0/312330392 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f00a0107700 0x7f00a0199910 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:59.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:59.453+0000 7f00a5be1700 1 --2- 172.21.15.32:0/312330392 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f00a0102620 0x7f00a019f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:59.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:59.453+0000 7f00a5be1700 1 --2- 172.21.15.32:0/312330392 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f00a0102fd0 0x7f00a019fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:44:59.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:59.453+0000 7f00a5be1700 1 -- 172.21.15.32:0/312330392 >> 172.21.15.32:0/312330392 conn(0x7f00a0079440 msgr2=0x7f00a0100350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:44:59.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:59.454+0000 7f00a5be1700 1 -- 172.21.15.32:0/312330392 shutdown_connections 2024-05-03T18:44:59.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:44:59.454+0000 7f00a5be1700 1 -- 172.21.15.32:0/312330392 wait complete. 2024-05-03T18:44:59.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:59 smithi032 ceph-mon[29763]: pgmap v284: 97 pgs: 97 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-05-03T18:44:59.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:59 smithi032 ceph-mon[29763]: from='client.? 172.21.15.32:0/312330392' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-03T18:44:59.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:44:59 smithi032 ceph-mon[29763]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-03T18:44:59.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:59 smithi045 ceph-mon[33299]: pgmap v284: 97 pgs: 97 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-05-03T18:44:59.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:59 smithi045 ceph-mon[33299]: from='client.? 172.21.15.32:0/312330392' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-03T18:44:59.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:44:59 smithi045 ceph-mon[33299]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-03T18:44:59.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:59 smithi113 ceph-mon[33540]: pgmap v284: 97 pgs: 97 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-05-03T18:44:59.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:59 smithi113 ceph-mon[33540]: from='client.? 172.21.15.32:0/312330392' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-03T18:44:59.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:44:59 smithi113 ceph-mon[33540]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-03T18:45:00.095 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-05-03T18:45:00.096 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid f500455e-097b-11ef-bc97-c7b262605968 -- ceph fs set cephfs max_mds 3 2024-05-03T18:45:00.342 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:45:00.747 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:45:00 smithi045 ceph-mon[33299]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-03T18:45:00.747 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:45:00 smithi045 ceph-mon[33299]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-03T18:45:00.788 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:45:00 smithi032 ceph-mon[29763]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-03T18:45:00.788 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:45:00 smithi032 ceph-mon[29763]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-03T18:45:00.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:45:00 smithi113 ceph-mon[33540]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-03T18:45:00.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:45:00 smithi113 ceph-mon[33540]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-03T18:45:01.656 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:45:01 smithi032 ceph-mon[29763]: pgmap v285: 97 pgs: 97 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 4 op/s 2024-05-03T18:45:01.747 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:45:01 smithi045 ceph-mon[33299]: pgmap v285: 97 pgs: 97 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 4 op/s 2024-05-03T18:45:01.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:45:01 smithi113 ceph-mon[33540]: pgmap v285: 97 pgs: 97 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 4 op/s 2024-05-03T18:45:01.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:01.878+0000 7f0ea9232700 1 -- 172.21.15.32:0/3415737422 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ea4108410 msgr2=0x7f0ea41087f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:01.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:01.878+0000 7f0ea9232700 1 --2- 172.21.15.32:0/3415737422 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ea4108410 0x7f0ea41087f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f0e94009990 tx=0x7f0e94009ca0 comp rx=0 tx=0).stop 2024-05-03T18:45:01.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:01.878+0000 7f0ea9232700 1 -- 172.21.15.32:0/3415737422 shutdown_connections 2024-05-03T18:45:01.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:01.878+0000 7f0ea9232700 1 --2- 172.21.15.32:0/3415737422 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0ea4102d60 0x7f0ea410f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:01.879 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:01.878+0000 7f0ea9232700 1 --2- 172.21.15.32:0/3415737422 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ea4108410 0x7f0ea41087f0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:01.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:01.878+0000 7f0ea9232700 1 --2- 172.21.15.32:0/3415737422 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0ea41023a0 0x7f0ea4102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:01.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:01.878+0000 7f0ea9232700 1 -- 172.21.15.32:0/3415737422 >> 172.21.15.32:0/3415737422 conn(0x7f0ea4079350 msgr2=0x7f0ea4079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:45:01.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:01.878+0000 7f0ea9232700 1 -- 172.21.15.32:0/3415737422 shutdown_connections 2024-05-03T18:45:01.880 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:01.878+0000 7f0ea9232700 1 -- 172.21.15.32:0/3415737422 wait complete. 2024-05-03T18:45:01.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:01.879+0000 7f0ea9232700 1 Processor -- start 2024-05-03T18:45:01.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:01.879+0000 7f0ea9232700 1 -- start start 2024-05-03T18:45:01.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:01.879+0000 7f0ea9232700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0ea41023a0 0x7f0ea419bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:01.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:01.879+0000 7f0ea9232700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ea4102d60 0x7f0ea419c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:01.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:01.879+0000 7f0ea9232700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0ea4108410 0x7f0ea41a0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:01.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:01.879+0000 7f0ea9232700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f0ea4112530 con 0x7f0ea4102d60 2024-05-03T18:45:01.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:01.879+0000 7f0ea9232700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f0ea41123b0 con 0x7f0ea41023a0 2024-05-03T18:45:01.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:01.879+0000 7f0ea9232700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f0ea41126b0 con 0x7f0ea4108410 2024-05-03T18:45:01.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:01.879+0000 7f0ea259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ea4102d60 0x7f0ea419c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:01.882 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:01.879+0000 7f0ea259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ea4102d60 0x7f0ea419c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45766/0 (socket says 172.21.15.32:45766) 2024-05-03T18:45:01.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:01.879+0000 7f0ea259c700 1 -- 172.21.15.32:0/3806911979 learned_addr learned my addr 172.21.15.32:0/3806911979 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:45:01.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:01.879+0000 7f0ea2d9d700 1 --2- 172.21.15.32:0/3806911979 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0ea41023a0 0x7f0ea419bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:01.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:01.879+0000 7f0ea359e700 1 --2- 172.21.15.32:0/3806911979 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0ea4108410 0x7f0ea41a0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:01.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:01.879+0000 7f0ea259c700 1 -- 172.21.15.32:0/3806911979 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0ea4108410 msgr2=0x7f0ea41a0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:01.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:01.879+0000 7f0ea259c700 1 --2- 172.21.15.32:0/3806911979 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0ea4108410 0x7f0ea41a0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:01.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:01.879+0000 7f0ea259c700 1 -- 172.21.15.32:0/3806911979 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0ea41023a0 msgr2=0x7f0ea419bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:01.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:01.879+0000 7f0ea259c700 1 --2- 172.21.15.32:0/3806911979 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0ea41023a0 0x7f0ea419bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:01.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:01.879+0000 7f0ea259c700 1 -- 172.21.15.32:0/3806911979 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ea41a0dd0 con 0x7f0ea4102d60 2024-05-03T18:45:01.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:01.880+0000 7f0ea2d9d700 1 --2- 172.21.15.32:0/3806911979 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0ea41023a0 0x7f0ea419bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T18:45:01.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:01.880+0000 7f0ea259c700 1 --2- 172.21.15.32:0/3806911979 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ea4102d60 0x7f0ea419c300 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f0e9800c920 tx=0x7f0e9800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:45:01.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:01.880+0000 7f0e8bfff700 1 -- 172.21.15.32:0/3806911979 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e98007b00 con 0x7f0ea4102d60 2024-05-03T18:45:01.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:01.880+0000 7f0ea9232700 1 -- 172.21.15.32:0/3806911979 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ea41a10c0 con 0x7f0ea4102d60 2024-05-03T18:45:01.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:01.880+0000 7f0ea9232700 1 -- 172.21.15.32:0/3806911979 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ea41a8940 con 0x7f0ea4102d60 2024-05-03T18:45:01.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:01.880+0000 7f0e8bfff700 1 -- 172.21.15.32:0/3806911979 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e98004ce0 con 0x7f0ea4102d60 2024-05-03T18:45:01.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:01.880+0000 7f0e8bfff700 1 -- 172.21.15.32:0/3806911979 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e98005650 con 0x7f0ea4102d60 2024-05-03T18:45:01.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:01.881+0000 7f0ea9232700 1 -- 172.21.15.32:0/3806911979 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e90005330 con 0x7f0ea4102d60 2024-05-03T18:45:01.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:01.882+0000 7f0e8bfff700 1 -- 172.21.15.32:0/3806911979 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f0e98007ca0 con 0x7f0ea4102d60 2024-05-03T18:45:01.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:01.883+0000 7f0e8bfff700 1 --2- 172.21.15.32:0/3806911979 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f0e8c0b2900 0x7f0e8c0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:01.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:01.883+0000 7f0ea2d9d700 1 --2- 172.21.15.32:0/3806911979 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f0e8c0b2900 0x7f0e8c0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:01.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:01.883+0000 7f0e8bfff700 1 -- 172.21.15.32:0/3806911979 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7979+0+0 (secure 0 0 0) 0x7f0e980e2120 con 0x7f0ea4102d60 2024-05-03T18:45:01.885 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:01.883+0000 7f0ea2d9d700 1 --2- 172.21.15.32:0/3806911979 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f0e8c0b2900 0x7f0e8c0b4dc0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f0e94006010 tx=0x7f0e9400b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:45:01.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:01.887+0000 7f0e8bfff700 1 -- 172.21.15.32:0/3806911979 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f0e98014030 con 0x7f0ea4102d60 2024-05-03T18:45:02.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:02.148+0000 7f0ea9232700 1 -- 172.21.15.32:0/3806911979 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7f0e900058f0 con 0x7f0ea4102d60 2024-05-03T18:45:02.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:02.459+0000 7f0e8bfff700 1 -- 172.21.15.32:0/3806911979 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v18) ==== 105+0+0 (secure 0 0 0) 0x7f0e980a9f70 con 0x7f0ea4102d60 2024-05-03T18:45:02.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:02.463+0000 7f0ea9232700 1 -- 172.21.15.32:0/3806911979 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f0e8c0b2900 msgr2=0x7f0e8c0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:02.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:02.463+0000 7f0ea9232700 1 --2- 172.21.15.32:0/3806911979 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f0e8c0b2900 0x7f0e8c0b4dc0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f0e94006010 tx=0x7f0e9400b540 comp rx=0 tx=0).stop 2024-05-03T18:45:02.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:02.463+0000 7f0ea9232700 1 -- 172.21.15.32:0/3806911979 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ea4102d60 msgr2=0x7f0ea419c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:02.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:02.463+0000 7f0ea9232700 1 --2- 172.21.15.32:0/3806911979 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ea4102d60 0x7f0ea419c300 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f0e9800c920 tx=0x7f0e9800cce0 comp rx=0 tx=0).stop 2024-05-03T18:45:02.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:02.463+0000 7f0ea9232700 1 -- 172.21.15.32:0/3806911979 shutdown_connections 2024-05-03T18:45:02.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:02.463+0000 7f0ea9232700 1 --2- 172.21.15.32:0/3806911979 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f0e8c0b2900 0x7f0e8c0b4dc0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:02.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:02.463+0000 7f0ea9232700 1 --2- 172.21.15.32:0/3806911979 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0ea4108410 0x7f0ea41a0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:02.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:02.463+0000 7f0ea9232700 1 --2- 172.21.15.32:0/3806911979 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0ea41023a0 0x7f0ea419bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:02.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:02.463+0000 7f0ea9232700 1 --2- 172.21.15.32:0/3806911979 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ea4102d60 0x7f0ea419c300 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:02.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:02.463+0000 7f0ea9232700 1 -- 172.21.15.32:0/3806911979 >> 172.21.15.32:0/3806911979 conn(0x7f0ea4079350 msgr2=0x7f0ea406b180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:45:02.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:02.463+0000 7f0ea9232700 1 -- 172.21.15.32:0/3806911979 shutdown_connections 2024-05-03T18:45:02.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:02.463+0000 7f0ea9232700 1 -- 172.21.15.32:0/3806911979 wait complete. 2024-05-03T18:45:02.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:45:02 smithi045 ceph-mon[33299]: from='client.? 172.21.15.32:0/3806911979' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-05-03T18:45:02.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:45:02 smithi113 ceph-mon[33540]: from='client.? 172.21.15.32:0/3806911979' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-05-03T18:45:02.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:45:02 smithi032 ceph-mon[29763]: from='client.? 172.21.15.32:0/3806911979' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-05-03T18:45:03.137 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid f500455e-097b-11ef-bc97-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-05-03T18:45:03.388 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:45:03.711 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:45:03 smithi032 ceph-mon[29763]: pgmap v286: 97 pgs: 97 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s wr, 4 op/s 2024-05-03T18:45:03.711 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:45:03 smithi032 ceph-mon[29763]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-03T18:45:03.711 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:45:03 smithi032 ceph-mon[29763]: from='client.? 172.21.15.32:0/3806911979' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-05-03T18:45:03.711 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:45:03 smithi032 ceph-mon[29763]: daemon mds.c assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-03T18:45:03.711 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:45:03 smithi032 ceph-mon[29763]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-03T18:45:03.711 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:45:03 smithi032 ceph-mon[29763]: fsmap cephfs:2 {0=i=up:active,1=c=up:creating} 10 up:standby 2024-05-03T18:45:03.711 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:45:03 smithi032 ceph-mon[29763]: daemon mds.c is now active in filesystem cephfs as rank 1 2024-05-03T18:45:03.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:45:03 smithi045 ceph-mon[33299]: pgmap v286: 97 pgs: 97 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s wr, 4 op/s 2024-05-03T18:45:03.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:45:03 smithi045 ceph-mon[33299]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-03T18:45:03.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:45:03 smithi045 ceph-mon[33299]: from='client.? 172.21.15.32:0/3806911979' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-05-03T18:45:03.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:45:03 smithi045 ceph-mon[33299]: daemon mds.c assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-03T18:45:03.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:45:03 smithi045 ceph-mon[33299]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-03T18:45:03.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:45:03 smithi045 ceph-mon[33299]: fsmap cephfs:2 {0=i=up:active,1=c=up:creating} 10 up:standby 2024-05-03T18:45:03.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:45:03 smithi045 ceph-mon[33299]: daemon mds.c is now active in filesystem cephfs as rank 1 2024-05-03T18:45:03.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:45:03 smithi113 ceph-mon[33540]: pgmap v286: 97 pgs: 97 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s wr, 4 op/s 2024-05-03T18:45:03.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:45:03 smithi113 ceph-mon[33540]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-03T18:45:03.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:45:03 smithi113 ceph-mon[33540]: from='client.? 172.21.15.32:0/3806911979' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-05-03T18:45:03.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:45:03 smithi113 ceph-mon[33540]: daemon mds.c assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-03T18:45:03.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:45:03 smithi113 ceph-mon[33540]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-03T18:45:03.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:45:03 smithi113 ceph-mon[33540]: fsmap cephfs:2 {0=i=up:active,1=c=up:creating} 10 up:standby 2024-05-03T18:45:03.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:45:03 smithi113 ceph-mon[33540]: daemon mds.c is now active in filesystem cephfs as rank 1 2024-05-03T18:45:04.700 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:45:04 smithi032 ceph-mon[29763]: mds.? [v2:172.21.15.113:6836/3677284460,v1:172.21.15.113:6837/3677284460] up:active 2024-05-03T18:45:04.700 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:45:04 smithi032 ceph-mon[29763]: daemon mds.f assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-03T18:45:04.700 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:45:04 smithi032 ceph-mon[29763]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-03T18:45:04.700 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:45:04 smithi032 ceph-mon[29763]: Cluster is now healthy 2024-05-03T18:45:04.700 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:45:04 smithi032 ceph-mon[29763]: fsmap cephfs:2 {0=i=up:active,1=c=up:active} 10 up:standby 2024-05-03T18:45:04.700 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:45:04 smithi032 ceph-mon[29763]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=f=up:creating} 9 up:standby 2024-05-03T18:45:04.700 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:45:04 smithi032 ceph-mon[29763]: daemon mds.f is now active in filesystem cephfs as rank 2 2024-05-03T18:45:04.747 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:45:04 smithi045 ceph-mon[33299]: mds.? [v2:172.21.15.113:6836/3677284460,v1:172.21.15.113:6837/3677284460] up:active 2024-05-03T18:45:04.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:45:04 smithi045 ceph-mon[33299]: daemon mds.f assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-03T18:45:04.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:45:04 smithi045 ceph-mon[33299]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-03T18:45:04.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:45:04 smithi045 ceph-mon[33299]: Cluster is now healthy 2024-05-03T18:45:04.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:45:04 smithi045 ceph-mon[33299]: fsmap cephfs:2 {0=i=up:active,1=c=up:active} 10 up:standby 2024-05-03T18:45:04.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:45:04 smithi045 ceph-mon[33299]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=f=up:creating} 9 up:standby 2024-05-03T18:45:04.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:45:04 smithi045 ceph-mon[33299]: daemon mds.f is now active in filesystem cephfs as rank 2 2024-05-03T18:45:04.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:45:04 smithi113 ceph-mon[33540]: mds.? [v2:172.21.15.113:6836/3677284460,v1:172.21.15.113:6837/3677284460] up:active 2024-05-03T18:45:04.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:45:04 smithi113 ceph-mon[33540]: daemon mds.f assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-03T18:45:04.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:45:04 smithi113 ceph-mon[33540]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-03T18:45:04.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:45:04 smithi113 ceph-mon[33540]: Cluster is now healthy 2024-05-03T18:45:04.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:45:04 smithi113 ceph-mon[33540]: fsmap cephfs:2 {0=i=up:active,1=c=up:active} 10 up:standby 2024-05-03T18:45:04.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:45:04 smithi113 ceph-mon[33540]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=f=up:creating} 9 up:standby 2024-05-03T18:45:04.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:45:04 smithi113 ceph-mon[33540]: daemon mds.f is now active in filesystem cephfs as rank 2 2024-05-03T18:45:04.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:04.916+0000 7fc1539fe700 1 -- 172.21.15.32:0/3074943032 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc14c06b180 msgr2=0x7fc14c06b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:04.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:04.916+0000 7fc1539fe700 1 --2- 172.21.15.32:0/3074943032 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc14c06b180 0x7fc14c06b600 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fc148009990 tx=0x7fc148009ca0 comp rx=0 tx=0).stop 2024-05-03T18:45:04.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:04.916+0000 7fc1539fe700 1 -- 172.21.15.32:0/3074943032 shutdown_connections 2024-05-03T18:45:04.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:04.916+0000 7fc1539fe700 1 --2- 172.21.15.32:0/3074943032 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc14c06bb40 0x7fc14c110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:04.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:04.916+0000 7fc1539fe700 1 --2- 172.21.15.32:0/3074943032 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc14c109510 0x7fc14c1098f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:04.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:04.916+0000 7fc1539fe700 1 --2- 172.21.15.32:0/3074943032 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc14c06b180 0x7fc14c06b600 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:04.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:04.916+0000 7fc1539fe700 1 -- 172.21.15.32:0/3074943032 >> 172.21.15.32:0/3074943032 conn(0x7fc14c079350 msgr2=0x7fc14c079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:45:04.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:04.916+0000 7fc1539fe700 1 -- 172.21.15.32:0/3074943032 shutdown_connections 2024-05-03T18:45:04.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:04.916+0000 7fc1539fe700 1 -- 172.21.15.32:0/3074943032 wait complete. 2024-05-03T18:45:04.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:04.917+0000 7fc1539fe700 1 Processor -- start 2024-05-03T18:45:04.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:04.917+0000 7fc1539fe700 1 -- start start 2024-05-03T18:45:04.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:04.917+0000 7fc1539fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc14c06b180 0x7fc14c19f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:04.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:04.917+0000 7fc1539fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc14c06bb40 0x7fc14c19fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:04.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:04.917+0000 7fc1539fe700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc14c109510 0x7fc14c199980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:04.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:04.917+0000 7fc1539fe700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fc14c113640 con 0x7fc14c06bb40 2024-05-03T18:45:04.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:04.917+0000 7fc1539fe700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc14c1134c0 con 0x7fc14c06b180 2024-05-03T18:45:04.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:04.917+0000 7fc1539fe700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fc14c1137c0 con 0x7fc14c109510 2024-05-03T18:45:04.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:04.917+0000 7fc150f99700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc14c06bb40 0x7fc14c19fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:04.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:04.917+0000 7fc150f99700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc14c06bb40 0x7fc14c19fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43400/0 (socket says 172.21.15.32:43400) 2024-05-03T18:45:04.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:04.917+0000 7fc150f99700 1 -- 172.21.15.32:0/1963920451 learned_addr learned my addr 172.21.15.32:0/1963920451 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:45:04.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:04.917+0000 7fc15179a700 1 --2- 172.21.15.32:0/1963920451 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc14c06b180 0x7fc14c19f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:04.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:04.917+0000 7fc151f9b700 1 --2- 172.21.15.32:0/1963920451 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc14c109510 0x7fc14c199980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:04.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:04.917+0000 7fc150f99700 1 -- 172.21.15.32:0/1963920451 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc14c109510 msgr2=0x7fc14c199980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:04.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:04.917+0000 7fc150f99700 1 --2- 172.21.15.32:0/1963920451 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc14c109510 0x7fc14c199980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:04.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:04.917+0000 7fc150f99700 1 -- 172.21.15.32:0/1963920451 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc14c06b180 msgr2=0x7fc14c19f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:04.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:04.917+0000 7fc150f99700 1 --2- 172.21.15.32:0/1963920451 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc14c06b180 0x7fc14c19f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:04.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:04.917+0000 7fc150f99700 1 -- 172.21.15.32:0/1963920451 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc14c19a0e0 con 0x7fc14c06bb40 2024-05-03T18:45:04.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:04.918+0000 7fc150f99700 1 --2- 172.21.15.32:0/1963920451 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc14c06bb40 0x7fc14c19fd00 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc148000c00 tx=0x7fc148009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:45:04.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:04.918+0000 7fc13e7fc700 1 -- 172.21.15.32:0/1963920451 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc148027070 con 0x7fc14c06bb40 2024-05-03T18:45:04.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:04.918+0000 7fc13e7fc700 1 -- 172.21.15.32:0/1963920451 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc148022470 con 0x7fc14c06bb40 2024-05-03T18:45:04.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:04.918+0000 7fc1539fe700 1 -- 172.21.15.32:0/1963920451 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc14c19a370 con 0x7fc14c06bb40 2024-05-03T18:45:04.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:04.918+0000 7fc13e7fc700 1 -- 172.21.15.32:0/1963920451 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc148017650 con 0x7fc14c06bb40 2024-05-03T18:45:04.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:04.918+0000 7fc1539fe700 1 -- 172.21.15.32:0/1963920451 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc14c1a6690 con 0x7fc14c06bb40 2024-05-03T18:45:04.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:04.919+0000 7fc1539fe700 1 -- 172.21.15.32:0/1963920451 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc14c06a190 con 0x7fc14c06bb40 2024-05-03T18:45:04.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:04.920+0000 7fc13e7fc700 1 -- 172.21.15.32:0/1963920451 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fc14802a070 con 0x7fc14c06bb40 2024-05-03T18:45:04.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:04.921+0000 7fc13e7fc700 1 --2- 172.21.15.32:0/1963920451 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fc1380b2880 0x7fc1380b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:04.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:04.921+0000 7fc15179a700 1 --2- 172.21.15.32:0/1963920451 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fc1380b2880 0x7fc1380b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:04.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:04.921+0000 7fc13e7fc700 1 -- 172.21.15.32:0/1963920451 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7979+0+0 (secure 0 0 0) 0x7fc1480e2ec0 con 0x7fc14c06bb40 2024-05-03T18:45:04.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:04.921+0000 7fc15179a700 1 --2- 172.21.15.32:0/1963920451 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fc1380b2880 0x7fc1380b4d40 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc14c19aac0 tx=0x7fc14000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:45:04.927 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:04.926+0000 7fc13e7fc700 1 -- 172.21.15.32:0/1963920451 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fc148013070 con 0x7fc14c06bb40 2024-05-03T18:45:05.193 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:05.191+0000 7fc1539fe700 1 -- 172.21.15.32:0/1963920451 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7fc14c02e510 con 0x7fc14c06bb40 2024-05-03T18:45:05.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:05.476+0000 7fc13e7fc700 1 -- 172.21.15.32:0/1963920451 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v23) ==== 121+0+0 (secure 0 0 0) 0x7fc1480aad20 con 0x7fc14c06bb40 2024-05-03T18:45:05.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:05.480+0000 7fc1539fe700 1 -- 172.21.15.32:0/1963920451 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fc1380b2880 msgr2=0x7fc1380b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:05.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:05.480+0000 7fc1539fe700 1 --2- 172.21.15.32:0/1963920451 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fc1380b2880 0x7fc1380b4d40 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc14c19aac0 tx=0x7fc14000b410 comp rx=0 tx=0).stop 2024-05-03T18:45:05.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:05.480+0000 7fc1539fe700 1 -- 172.21.15.32:0/1963920451 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc14c06bb40 msgr2=0x7fc14c19fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:05.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:05.480+0000 7fc1539fe700 1 --2- 172.21.15.32:0/1963920451 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc14c06bb40 0x7fc14c19fd00 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc148000c00 tx=0x7fc148009580 comp rx=0 tx=0).stop 2024-05-03T18:45:05.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:05.480+0000 7fc1539fe700 1 -- 172.21.15.32:0/1963920451 shutdown_connections 2024-05-03T18:45:05.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:05.480+0000 7fc1539fe700 1 --2- 172.21.15.32:0/1963920451 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fc1380b2880 0x7fc1380b4d40 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:05.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:05.480+0000 7fc1539fe700 1 --2- 172.21.15.32:0/1963920451 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc14c109510 0x7fc14c199980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:05.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:05.480+0000 7fc1539fe700 1 --2- 172.21.15.32:0/1963920451 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc14c06b180 0x7fc14c19f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:05.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:05.480+0000 7fc1539fe700 1 --2- 172.21.15.32:0/1963920451 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc14c06bb40 0x7fc14c19fd00 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:05.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:05.480+0000 7fc1539fe700 1 -- 172.21.15.32:0/1963920451 >> 172.21.15.32:0/1963920451 conn(0x7fc14c079350 msgr2=0x7fc14c1010d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:45:05.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:05.480+0000 7fc1539fe700 1 -- 172.21.15.32:0/1963920451 shutdown_connections 2024-05-03T18:45:05.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:05.480+0000 7fc1539fe700 1 -- 172.21.15.32:0/1963920451 wait complete. 2024-05-03T18:45:05.747 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:45:05 smithi045 ceph-mon[33299]: pgmap v287: 97 pgs: 97 active+clean; 580 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 5 op/s 2024-05-03T18:45:05.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:45:05 smithi045 ceph-mon[33299]: mds.? [v2:172.21.15.113:6834/2588738276,v1:172.21.15.113:6835/2588738276] up:active 2024-05-03T18:45:05.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:45:05 smithi045 ceph-mon[33299]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=f=up:active} 9 up:standby 2024-05-03T18:45:05.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:45:05 smithi045 ceph-mon[33299]: from='client.? 172.21.15.32:0/1963920451' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-03T18:45:05.796 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:45:05 smithi032 ceph-mon[29763]: pgmap v287: 97 pgs: 97 active+clean; 580 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 5 op/s 2024-05-03T18:45:05.796 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:45:05 smithi032 ceph-mon[29763]: mds.? [v2:172.21.15.113:6834/2588738276,v1:172.21.15.113:6835/2588738276] up:active 2024-05-03T18:45:05.796 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:45:05 smithi032 ceph-mon[29763]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=f=up:active} 9 up:standby 2024-05-03T18:45:05.796 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:45:05 smithi032 ceph-mon[29763]: from='client.? 172.21.15.32:0/1963920451' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-03T18:45:05.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:45:05 smithi113 ceph-mon[33540]: pgmap v287: 97 pgs: 97 active+clean; 580 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 5 op/s 2024-05-03T18:45:05.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:45:05 smithi113 ceph-mon[33540]: mds.? [v2:172.21.15.113:6834/2588738276,v1:172.21.15.113:6835/2588738276] up:active 2024-05-03T18:45:05.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:45:05 smithi113 ceph-mon[33540]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=f=up:active} 9 up:standby 2024-05-03T18:45:05.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:45:05 smithi113 ceph-mon[33540]: from='client.? 172.21.15.32:0/1963920451' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-03T18:45:06.249 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid f500455e-097b-11ef-bc97-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-05-03T18:45:06.505 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:45:06.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:45:06 smithi045 ceph-mon[33299]: from='client.? 172.21.15.32:0/1963920451' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-03T18:45:06.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:45:06 smithi045 ceph-mon[33299]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=f=up:active} 9 up:standby 2024-05-03T18:45:06.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:45:06 smithi045 ceph-mon[33299]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2024-05-03T18:45:06.791 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:45:06 smithi032 ceph-mon[29763]: from='client.? 172.21.15.32:0/1963920451' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-03T18:45:06.791 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:45:06 smithi032 ceph-mon[29763]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=f=up:active} 9 up:standby 2024-05-03T18:45:06.791 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:45:06 smithi032 ceph-mon[29763]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2024-05-03T18:45:06.800 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:45:06 smithi113 ceph-mon[33540]: from='client.? 172.21.15.32:0/1963920451' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-03T18:45:06.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:45:06 smithi113 ceph-mon[33540]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=f=up:active} 9 up:standby 2024-05-03T18:45:06.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:45:06 smithi113 ceph-mon[33540]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2024-05-03T18:45:07.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:45:07 smithi045 ceph-mon[33299]: pgmap v288: 97 pgs: 97 active+clean; 581 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s rd, 2.1 KiB/s wr, 7 op/s 2024-05-03T18:45:07.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:45:07 smithi113 ceph-mon[33540]: pgmap v288: 97 pgs: 97 active+clean; 581 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s rd, 2.1 KiB/s wr, 7 op/s 2024-05-03T18:45:07.801 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:45:07 smithi032 ceph-mon[29763]: pgmap v288: 97 pgs: 97 active+clean; 581 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s rd, 2.1 KiB/s wr, 7 op/s 2024-05-03T18:45:08.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:08.122+0000 7fc81abb8700 1 -- 172.21.15.32:0/2994350235 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc814108850 msgr2=0x7fc8141103d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:08.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:08.122+0000 7fc81abb8700 1 --2- 172.21.15.32:0/2994350235 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc814108850 0x7fc8141103d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc80c00b0d0 tx=0x7fc80c00b3e0 comp rx=0 tx=0).stop 2024-05-03T18:45:08.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:08.122+0000 7fc81abb8700 1 -- 172.21.15.32:0/2994350235 shutdown_connections 2024-05-03T18:45:08.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:08.122+0000 7fc81abb8700 1 --2- 172.21.15.32:0/2994350235 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc814108850 0x7fc8141103d0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:08.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:08.122+0000 7fc81abb8700 1 --2- 172.21.15.32:0/2994350235 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc814103750 0x7fc814103b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:08.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:08.122+0000 7fc81abb8700 1 --2- 172.21.15.32:0/2994350235 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc814104100 0x7fc814108120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:08.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:08.122+0000 7fc81abb8700 1 -- 172.21.15.32:0/2994350235 >> 172.21.15.32:0/2994350235 conn(0x7fc81407a600 msgr2=0x7fc81407aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:45:08.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:08.122+0000 7fc81abb8700 1 -- 172.21.15.32:0/2994350235 shutdown_connections 2024-05-03T18:45:08.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:08.122+0000 7fc81abb8700 1 -- 172.21.15.32:0/2994350235 wait complete. 2024-05-03T18:45:08.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:08.123+0000 7fc81abb8700 1 Processor -- start 2024-05-03T18:45:08.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:08.123+0000 7fc81abb8700 1 -- start start 2024-05-03T18:45:08.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:08.123+0000 7fc81abb8700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc814103750 0x7fc8140757a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:08.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:08.123+0000 7fc81abb8700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc814104100 0x7fc814075ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:08.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:08.123+0000 7fc81abb8700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc814108850 0x7fc814076220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:08.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:08.123+0000 7fc81abb8700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fc814113690 con 0x7fc814103750 2024-05-03T18:45:08.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:08.123+0000 7fc81abb8700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc814113510 con 0x7fc814108850 2024-05-03T18:45:08.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:08.123+0000 7fc81abb8700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fc814113810 con 0x7fc814104100 2024-05-03T18:45:08.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:08.123+0000 7fc818954700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc814103750 0x7fc8140757a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:08.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:08.123+0000 7fc818954700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc814103750 0x7fc8140757a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43428/0 (socket says 172.21.15.32:43428) 2024-05-03T18:45:08.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:08.123+0000 7fc818954700 1 -- 172.21.15.32:0/31534933 learned_addr learned my addr 172.21.15.32:0/31534933 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:45:08.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:08.123+0000 7fc819155700 1 --2- 172.21.15.32:0/31534933 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc814108850 0x7fc814076220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:08.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:08.123+0000 7fc813fff700 1 --2- 172.21.15.32:0/31534933 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc814104100 0x7fc814075ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:08.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:08.123+0000 7fc818954700 1 -- 172.21.15.32:0/31534933 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc814104100 msgr2=0x7fc814075ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:08.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:08.123+0000 7fc818954700 1 --2- 172.21.15.32:0/31534933 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc814104100 0x7fc814075ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:08.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:08.123+0000 7fc818954700 1 -- 172.21.15.32:0/31534933 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc814108850 msgr2=0x7fc814076220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:08.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:08.123+0000 7fc818954700 1 --2- 172.21.15.32:0/31534933 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc814108850 0x7fc814076220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:08.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:08.123+0000 7fc818954700 1 -- 172.21.15.32:0/31534933 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8141a7ff0 con 0x7fc814103750 2024-05-03T18:45:08.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:08.123+0000 7fc819155700 1 --2- 172.21.15.32:0/31534933 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc814108850 0x7fc814076220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T18:45:08.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:08.124+0000 7fc818954700 1 --2- 172.21.15.32:0/31534933 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc814103750 0x7fc8140757a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc80400d950 tx=0x7fc80400dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:45:08.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:08.124+0000 7fc811ffb700 1 -- 172.21.15.32:0/31534933 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc804016070 con 0x7fc814103750 2024-05-03T18:45:08.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:08.124+0000 7fc81abb8700 1 -- 172.21.15.32:0/31534933 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8141a82e0 con 0x7fc814103750 2024-05-03T18:45:08.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:08.124+0000 7fc811ffb700 1 -- 172.21.15.32:0/31534933 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc804010460 con 0x7fc814103750 2024-05-03T18:45:08.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:08.124+0000 7fc811ffb700 1 -- 172.21.15.32:0/31534933 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc80400f620 con 0x7fc814103750 2024-05-03T18:45:08.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:08.124+0000 7fc81abb8700 1 -- 172.21.15.32:0/31534933 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8141a8870 con 0x7fc814103750 2024-05-03T18:45:08.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:08.125+0000 7fc81abb8700 1 -- 172.21.15.32:0/31534933 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc81406a190 con 0x7fc814103750 2024-05-03T18:45:08.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:08.126+0000 7fc811ffb700 1 -- 172.21.15.32:0/31534933 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fc804019070 con 0x7fc814103750 2024-05-03T18:45:08.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:08.127+0000 7fc811ffb700 1 --2- 172.21.15.32:0/31534933 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fc7fc0b2880 0x7fc7fc0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:08.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:08.127+0000 7fc813fff700 1 --2- 172.21.15.32:0/31534933 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fc7fc0b2880 0x7fc7fc0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:08.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:08.127+0000 7fc811ffb700 1 -- 172.21.15.32:0/31534933 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7979+0+0 (secure 0 0 0) 0x7fc8040e2130 con 0x7fc814103750 2024-05-03T18:45:08.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:08.127+0000 7fc813fff700 1 --2- 172.21.15.32:0/31534933 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fc7fc0b2880 0x7fc7fc0b4d40 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fc808005fd0 tx=0x7fc808005ee0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:45:08.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:08.131+0000 7fc811ffb700 1 -- 172.21.15.32:0/31534933 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fc8040a9f00 con 0x7fc814103750 2024-05-03T18:45:08.404 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:08.402+0000 7fc81abb8700 1 -- 172.21.15.32:0/31534933 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7fc81402e730 con 0x7fc814103750 2024-05-03T18:45:08.496 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:08.494+0000 7fc811ffb700 1 -- 172.21.15.32:0/31534933 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v25) ==== 115+0+0 (secure 0 0 0) 0x7fc8040a9800 con 0x7fc814103750 2024-05-03T18:45:08.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:08.496+0000 7fc81abb8700 1 -- 172.21.15.32:0/31534933 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fc7fc0b2880 msgr2=0x7fc7fc0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:08.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:08.496+0000 7fc81abb8700 1 --2- 172.21.15.32:0/31534933 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fc7fc0b2880 0x7fc7fc0b4d40 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fc808005fd0 tx=0x7fc808005ee0 comp rx=0 tx=0).stop 2024-05-03T18:45:08.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:08.496+0000 7fc81abb8700 1 -- 172.21.15.32:0/31534933 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc814103750 msgr2=0x7fc8140757a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:08.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:08.496+0000 7fc81abb8700 1 --2- 172.21.15.32:0/31534933 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc814103750 0x7fc8140757a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc80400d950 tx=0x7fc80400dd10 comp rx=0 tx=0).stop 2024-05-03T18:45:08.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:08.497+0000 7fc81abb8700 1 -- 172.21.15.32:0/31534933 shutdown_connections 2024-05-03T18:45:08.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:08.497+0000 7fc81abb8700 1 --2- 172.21.15.32:0/31534933 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fc7fc0b2880 0x7fc7fc0b4d40 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:08.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:08.497+0000 7fc81abb8700 1 --2- 172.21.15.32:0/31534933 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc814108850 0x7fc814076220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:08.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:08.497+0000 7fc81abb8700 1 --2- 172.21.15.32:0/31534933 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc814103750 0x7fc8140757a0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:08.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:08.497+0000 7fc81abb8700 1 --2- 172.21.15.32:0/31534933 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc814104100 0x7fc814075ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:08.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:08.497+0000 7fc81abb8700 1 -- 172.21.15.32:0/31534933 >> 172.21.15.32:0/31534933 conn(0x7fc81407a600 msgr2=0x7fc814101720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:45:08.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:08.497+0000 7fc81abb8700 1 -- 172.21.15.32:0/31534933 shutdown_connections 2024-05-03T18:45:08.500 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:08.497+0000 7fc81abb8700 1 -- 172.21.15.32:0/31534933 wait complete. 2024-05-03T18:45:08.747 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:45:08 smithi045 ceph-mon[33299]: from='client.? 172.21.15.32:0/31534933' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-03T18:45:08.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:45:08 smithi113 ceph-mon[33540]: from='client.? 172.21.15.32:0/31534933' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-03T18:45:08.808 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:45:08 smithi032 ceph-mon[29763]: from='client.? 172.21.15.32:0/31534933' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-03T18:45:09.151 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated'} subvols for filesystem 'cephfs' 2024-05-03T18:45:09.152 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid f500455e-097b-11ef-bc97-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-05-03T18:45:09.402 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:45:09.755 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:45:09 smithi032 ceph-mon[29763]: pgmap v289: 97 pgs: 97 active+clean; 582 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2.1 KiB/s wr, 8 op/s 2024-05-03T18:45:09.756 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:45:09 smithi032 ceph-mon[29763]: from='client.? 172.21.15.32:0/31534933' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-03T18:45:09.756 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:45:09 smithi032 ceph-mon[29763]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2024-05-03T18:45:09.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:45:09 smithi113 ceph-mon[33540]: pgmap v289: 97 pgs: 97 active+clean; 582 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2.1 KiB/s wr, 8 op/s 2024-05-03T18:45:09.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:45:09 smithi113 ceph-mon[33540]: from='client.? 172.21.15.32:0/31534933' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-03T18:45:09.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:45:09 smithi113 ceph-mon[33540]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2024-05-03T18:45:09.997 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:45:09 smithi045 ceph-mon[33299]: pgmap v289: 97 pgs: 97 active+clean; 582 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2.1 KiB/s wr, 8 op/s 2024-05-03T18:45:09.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:45:09 smithi045 ceph-mon[33299]: from='client.? 172.21.15.32:0/31534933' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-03T18:45:09.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:45:09 smithi045 ceph-mon[33299]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2024-05-03T18:45:10.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:10.853+0000 7fc233561700 1 -- 172.21.15.32:0/3798125188 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc22c06c590 msgr2=0x7fc22c110380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:10.856 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:10.853+0000 7fc233561700 1 --2- 172.21.15.32:0/3798125188 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc22c06c590 0x7fc22c110380 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fc22400b0d0 tx=0x7fc22400b3e0 comp rx=0 tx=0).stop 2024-05-03T18:45:10.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:10.853+0000 7fc233561700 1 -- 172.21.15.32:0/3798125188 shutdown_connections 2024-05-03T18:45:10.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:10.853+0000 7fc233561700 1 --2- 172.21.15.32:0/3798125188 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc22c06c590 0x7fc22c110380 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:10.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:10.853+0000 7fc233561700 1 --2- 172.21.15.32:0/3798125188 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc22c1095b0 0x7fc22c109990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:10.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:10.853+0000 7fc233561700 1 --2- 172.21.15.32:0/3798125188 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc22c06bbd0 0x7fc22c06c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:10.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:10.853+0000 7fc233561700 1 -- 172.21.15.32:0/3798125188 >> 172.21.15.32:0/3798125188 conn(0x7fc22c078210 msgr2=0x7fc22c078620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:45:10.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:10.853+0000 7fc233561700 1 -- 172.21.15.32:0/3798125188 shutdown_connections 2024-05-03T18:45:10.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:10.853+0000 7fc233561700 1 -- 172.21.15.32:0/3798125188 wait complete. 2024-05-03T18:45:10.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:10.854+0000 7fc233561700 1 Processor -- start 2024-05-03T18:45:10.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:10.854+0000 7fc233561700 1 -- start start 2024-05-03T18:45:10.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:10.854+0000 7fc233561700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc22c06bbd0 0x7fc22c1027c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:10.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:10.854+0000 7fc233561700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc22c06c590 0x7fc22c102d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:10.857 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:10.854+0000 7fc233561700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc22c1095b0 0x7fc22c103240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:10.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:10.854+0000 7fc233561700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fc22c1136a0 con 0x7fc22c1095b0 2024-05-03T18:45:10.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:10.854+0000 7fc233561700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc22c113520 con 0x7fc22c06bbd0 2024-05-03T18:45:10.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:10.854+0000 7fc233561700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fc22c113820 con 0x7fc22c06c590 2024-05-03T18:45:10.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:10.854+0000 7fc231afe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc22c1095b0 0x7fc22c103240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:10.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:10.854+0000 7fc231afe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc22c1095b0 0x7fc22c103240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43458/0 (socket says 172.21.15.32:43458) 2024-05-03T18:45:10.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:10.854+0000 7fc231afe700 1 -- 172.21.15.32:0/2371149664 learned_addr learned my addr 172.21.15.32:0/2371149664 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:45:10.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:10.854+0000 7fc2312fd700 1 --2- 172.21.15.32:0/2371149664 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc22c06bbd0 0x7fc22c1027c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:10.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:10.854+0000 7fc230afc700 1 --2- 172.21.15.32:0/2371149664 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc22c06c590 0x7fc22c102d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:10.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:10.854+0000 7fc231afe700 1 -- 172.21.15.32:0/2371149664 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc22c06c590 msgr2=0x7fc22c102d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:10.858 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:10.854+0000 7fc231afe700 1 --2- 172.21.15.32:0/2371149664 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc22c06c590 0x7fc22c102d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:10.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:10.854+0000 7fc231afe700 1 -- 172.21.15.32:0/2371149664 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc22c06bbd0 msgr2=0x7fc22c1027c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:10.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:10.854+0000 7fc231afe700 1 --2- 172.21.15.32:0/2371149664 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc22c06bbd0 0x7fc22c1027c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:10.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:10.854+0000 7fc231afe700 1 -- 172.21.15.32:0/2371149664 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc22c1a7f30 con 0x7fc22c1095b0 2024-05-03T18:45:10.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:10.854+0000 7fc230afc700 1 --2- 172.21.15.32:0/2371149664 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc22c06c590 0x7fc22c102d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:45:10.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:10.854+0000 7fc231afe700 1 --2- 172.21.15.32:0/2371149664 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc22c1095b0 0x7fc22c103240 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fc224009fd0 tx=0x7fc224007ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:45:10.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:10.855+0000 7fc2227fc700 1 -- 172.21.15.32:0/2371149664 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc22401f070 con 0x7fc22c1095b0 2024-05-03T18:45:10.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:10.855+0000 7fc233561700 1 -- 172.21.15.32:0/2371149664 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc22c1a81c0 con 0x7fc22c1095b0 2024-05-03T18:45:10.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:10.855+0000 7fc2227fc700 1 -- 172.21.15.32:0/2371149664 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc224003c00 con 0x7fc22c1095b0 2024-05-03T18:45:10.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:10.855+0000 7fc2227fc700 1 -- 172.21.15.32:0/2371149664 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc22401b8e0 con 0x7fc22c1095b0 2024-05-03T18:45:10.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:10.855+0000 7fc233561700 1 -- 172.21.15.32:0/2371149664 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc22c1a8670 con 0x7fc22c1095b0 2024-05-03T18:45:10.859 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:10.856+0000 7fc233561700 1 -- 172.21.15.32:0/2371149664 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc22c104c00 con 0x7fc22c1095b0 2024-05-03T18:45:10.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:10.857+0000 7fc2227fc700 1 -- 172.21.15.32:0/2371149664 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fc224022070 con 0x7fc22c1095b0 2024-05-03T18:45:10.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:10.858+0000 7fc2227fc700 1 --2- 172.21.15.32:0/2371149664 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fc2180b2880 0x7fc2180b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:10.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:10.858+0000 7fc2312fd700 1 --2- 172.21.15.32:0/2371149664 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fc2180b2880 0x7fc2180b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:10.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:10.858+0000 7fc2227fc700 1 -- 172.21.15.32:0/2371149664 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7979+0+0 (secure 0 0 0) 0x7fc22400f020 con 0x7fc22c1095b0 2024-05-03T18:45:10.860 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:10.858+0000 7fc2312fd700 1 --2- 172.21.15.32:0/2371149664 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fc2180b2880 0x7fc2180b4d40 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc21c005fd0 tx=0x7fc21c00af20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:45:10.862 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:10.861+0000 7fc2227fc700 1 -- 172.21.15.32:0/2371149664 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fc224029020 con 0x7fc22c1095b0 2024-05-03T18:45:11.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:11.107+0000 7fc233561700 1 -- 172.21.15.32:0/2371149664 --> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fc22c1017b0 con 0x7fc2180b2880 2024-05-03T18:45:11.135 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:11.134+0000 7fc2227fc700 1 -- 172.21.15.32:0/2371149664 <== mgr.14152 v2:172.21.15.32:6800/1796547904 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fc22c1017b0 con 0x7fc2180b2880 2024-05-03T18:45:11.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:11.136+0000 7fc233561700 1 -- 172.21.15.32:0/2371149664 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fc2180b2880 msgr2=0x7fc2180b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:11.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:11.136+0000 7fc233561700 1 --2- 172.21.15.32:0/2371149664 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fc2180b2880 0x7fc2180b4d40 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc21c005fd0 tx=0x7fc21c00af20 comp rx=0 tx=0).stop 2024-05-03T18:45:11.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:11.136+0000 7fc233561700 1 -- 172.21.15.32:0/2371149664 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc22c1095b0 msgr2=0x7fc22c103240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:11.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:11.136+0000 7fc233561700 1 --2- 172.21.15.32:0/2371149664 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc22c1095b0 0x7fc22c103240 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fc224009fd0 tx=0x7fc224007ad0 comp rx=0 tx=0).stop 2024-05-03T18:45:11.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:11.136+0000 7fc233561700 1 -- 172.21.15.32:0/2371149664 shutdown_connections 2024-05-03T18:45:11.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:11.136+0000 7fc233561700 1 --2- 172.21.15.32:0/2371149664 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fc2180b2880 0x7fc2180b4d40 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:11.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:11.136+0000 7fc233561700 1 --2- 172.21.15.32:0/2371149664 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc22c1095b0 0x7fc22c103240 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:11.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:11.136+0000 7fc233561700 1 --2- 172.21.15.32:0/2371149664 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc22c06bbd0 0x7fc22c1027c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:11.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:11.136+0000 7fc233561700 1 --2- 172.21.15.32:0/2371149664 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc22c06c590 0x7fc22c102d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:11.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:11.136+0000 7fc233561700 1 -- 172.21.15.32:0/2371149664 >> 172.21.15.32:0/2371149664 conn(0x7fc22c078210 msgr2=0x7fc22c1010a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:45:11.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:11.136+0000 7fc233561700 1 -- 172.21.15.32:0/2371149664 shutdown_connections 2024-05-03T18:45:11.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:11.136+0000 7fc233561700 1 -- 172.21.15.32:0/2371149664 wait complete. 2024-05-03T18:45:11.434 INFO:journalctl@ceph.mgr.x.smithi032.stdout:May 03 18:45:11 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-x[29995]: 2024-05-03T18:45:11.113+0000 7fec980af700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-03T18:45:11.435 INFO:journalctl@ceph.mgr.x.smithi032.stdout:May 03 18:45:11 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-x[29995]: 2024-05-03T18:45:11.113+0000 7fec980af700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-03T18:45:11.435 INFO:journalctl@ceph.mgr.x.smithi032.stdout:May 03 18:45:11 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-x[29995]: 2024-05-03T18:45:11.113+0000 7fec980af700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-03T18:45:11.435 INFO:journalctl@ceph.mgr.x.smithi032.stdout:May 03 18:45:11 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-x[29995]: 2024-05-03T18:45:11.113+0000 7fec980af700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-03T18:45:11.435 INFO:journalctl@ceph.mgr.x.smithi032.stdout:May 03 18:45:11 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-x[29995]: 2024-05-03T18:45:11.113+0000 7fec980af700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-03T18:45:11.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:45:11 smithi113 ceph-mon[33540]: pgmap v290: 97 pgs: 97 active+clean; 582 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-05-03T18:45:11.809 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:45:11 smithi032 ceph-mon[29763]: pgmap v290: 97 pgs: 97 active+clean; 582 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-05-03T18:45:11.839 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid f500455e-097b-11ef-bc97-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-05-03T18:45:11.997 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:45:11 smithi045 ceph-mon[33299]: pgmap v290: 97 pgs: 97 active+clean; 582 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-05-03T18:45:12.087 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:45:12.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:45:12 smithi113 ceph-mon[33540]: from='client.14700 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:45:12.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:45:12 smithi032 ceph-mon[29763]: from='client.14700 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:45:12.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:45:12 smithi045 ceph-mon[33299]: from='client.14700 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:45:13.686 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:45:13 smithi032 ceph-mon[29763]: pgmap v291: 97 pgs: 97 active+clean; 582 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-05-03T18:45:13.687 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:45:13 smithi032 ceph-mon[29763]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-05-03T18:45:13.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:13.689+0000 7fcfc457c700 1 -- 172.21.15.32:0/842671578 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcfbc109500 msgr2=0x7fcfbc1098e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:13.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:13.689+0000 7fcfc457c700 1 --2- 172.21.15.32:0/842671578 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcfbc109500 0x7fcfbc1098e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fcfb8009990 tx=0x7fcfb8009ca0 comp rx=0 tx=0).stop 2024-05-03T18:45:13.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:13.689+0000 7fcfc457c700 1 -- 172.21.15.32:0/842671578 shutdown_connections 2024-05-03T18:45:13.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:13.689+0000 7fcfc457c700 1 --2- 172.21.15.32:0/842671578 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcfbc06c3d0 0x7fcfbc110320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:13.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:13.689+0000 7fcfc457c700 1 --2- 172.21.15.32:0/842671578 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcfbc109500 0x7fcfbc1098e0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:13.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:13.689+0000 7fcfc457c700 1 --2- 172.21.15.32:0/842671578 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcfbc06ba10 0x7fcfbc06be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:13.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:13.689+0000 7fcfc457c700 1 -- 172.21.15.32:0/842671578 >> 172.21.15.32:0/842671578 conn(0x7fcfbc079310 msgr2=0x7fcfbc079720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:45:13.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:13.689+0000 7fcfc457c700 1 -- 172.21.15.32:0/842671578 shutdown_connections 2024-05-03T18:45:13.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:13.689+0000 7fcfc457c700 1 -- 172.21.15.32:0/842671578 wait complete. 2024-05-03T18:45:13.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:13.689+0000 7fcfc457c700 1 Processor -- start 2024-05-03T18:45:13.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:13.690+0000 7fcfc457c700 1 -- start start 2024-05-03T18:45:13.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:13.690+0000 7fcfc457c700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcfbc06ba10 0x7fcfbc102460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:13.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:13.690+0000 7fcfc457c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcfbc06c3d0 0x7fcfbc1029a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:13.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:13.690+0000 7fcfc457c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcfbc109500 0x7fcfbc102ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:13.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:13.690+0000 7fcfc457c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fcfbc113630 con 0x7fcfbc06c3d0 2024-05-03T18:45:13.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:13.690+0000 7fcfc457c700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fcfbc1134b0 con 0x7fcfbc109500 2024-05-03T18:45:13.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:13.690+0000 7fcfc457c700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fcfbc1137b0 con 0x7fcfbc06ba10 2024-05-03T18:45:13.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:13.690+0000 7fcfc1b17700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcfbc06c3d0 0x7fcfbc1029a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:13.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:13.690+0000 7fcfc1b17700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcfbc06c3d0 0x7fcfbc1029a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48652/0 (socket says 172.21.15.32:48652) 2024-05-03T18:45:13.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:13.690+0000 7fcfc1b17700 1 -- 172.21.15.32:0/3528817839 learned_addr learned my addr 172.21.15.32:0/3528817839 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:45:13.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:13.690+0000 7fcfc2318700 1 --2- 172.21.15.32:0/3528817839 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcfbc06ba10 0x7fcfbc102460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:13.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:13.690+0000 7fcfc2b19700 1 --2- 172.21.15.32:0/3528817839 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcfbc109500 0x7fcfbc102ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:13.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:13.690+0000 7fcfc1b17700 1 -- 172.21.15.32:0/3528817839 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcfbc06ba10 msgr2=0x7fcfbc102460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:13.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:13.690+0000 7fcfc1b17700 1 --2- 172.21.15.32:0/3528817839 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcfbc06ba10 0x7fcfbc102460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:13.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:13.690+0000 7fcfc1b17700 1 -- 172.21.15.32:0/3528817839 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcfbc109500 msgr2=0x7fcfbc102ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:13.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:13.690+0000 7fcfc1b17700 1 --2- 172.21.15.32:0/3528817839 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcfbc109500 0x7fcfbc102ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:13.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:13.690+0000 7fcfc1b17700 1 -- 172.21.15.32:0/3528817839 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfbc1a8180 con 0x7fcfbc06c3d0 2024-05-03T18:45:13.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:13.690+0000 7fcfc1b17700 1 --2- 172.21.15.32:0/3528817839 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcfbc06c3d0 0x7fcfbc1029a0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fcfac00d950 tx=0x7fcfac00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:45:13.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:13.691+0000 7fcfb37fe700 1 -- 172.21.15.32:0/3528817839 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfac016070 con 0x7fcfbc06c3d0 2024-05-03T18:45:13.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:13.691+0000 7fcfc457c700 1 -- 172.21.15.32:0/3528817839 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfbc1a8470 con 0x7fcfbc06c3d0 2024-05-03T18:45:13.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:13.691+0000 7fcfb37fe700 1 -- 172.21.15.32:0/3528817839 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcfac00fd40 con 0x7fcfbc06c3d0 2024-05-03T18:45:13.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:13.691+0000 7fcfb37fe700 1 -- 172.21.15.32:0/3528817839 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfac00e3b0 con 0x7fcfbc06c3d0 2024-05-03T18:45:13.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:13.691+0000 7fcfc457c700 1 -- 172.21.15.32:0/3528817839 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfbc1a8a00 con 0x7fcfbc06c3d0 2024-05-03T18:45:13.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:13.692+0000 7fcfc457c700 1 -- 172.21.15.32:0/3528817839 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcfbc10b9e0 con 0x7fcfbc06c3d0 2024-05-03T18:45:13.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:13.693+0000 7fcfb37fe700 1 -- 172.21.15.32:0/3528817839 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fcfac019070 con 0x7fcfbc06c3d0 2024-05-03T18:45:13.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:13.694+0000 7fcfb37fe700 1 --2- 172.21.15.32:0/3528817839 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fcfa80b2910 0x7fcfa80b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:13.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:13.694+0000 7fcfc2318700 1 --2- 172.21.15.32:0/3528817839 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fcfa80b2910 0x7fcfa80b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:13.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:13.694+0000 7fcfb37fe700 1 -- 172.21.15.32:0/3528817839 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7979+0+0 (secure 0 0 0) 0x7fcfac0e1d40 con 0x7fcfbc06c3d0 2024-05-03T18:45:13.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:13.694+0000 7fcfc2318700 1 --2- 172.21.15.32:0/3528817839 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fcfa80b2910 0x7fcfa80b4dd0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fcfb800b5c0 tx=0x7fcfb8005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:45:13.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:13.697+0000 7fcfb37fe700 1 -- 172.21.15.32:0/3528817839 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fcfac0a99b0 con 0x7fcfbc06c3d0 2024-05-03T18:45:13.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:45:13 smithi113 ceph-mon[33540]: pgmap v291: 97 pgs: 97 active+clean; 582 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-05-03T18:45:13.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:45:13 smithi113 ceph-mon[33540]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-05-03T18:45:13.946 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:13.946+0000 7fcfc457c700 1 -- 172.21.15.32:0/3528817839 --> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fcfbc0638e0 con 0x7fcfa80b2910 2024-05-03T18:45:13.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:13.975+0000 7fcfb37fe700 1 -- 172.21.15.32:0/3528817839 <== mgr.14152 v2:172.21.15.32:6800/1796547904 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fcfbc0638e0 con 0x7fcfa80b2910 2024-05-03T18:45:13.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:13.977+0000 7fcfc457c700 1 -- 172.21.15.32:0/3528817839 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fcfa80b2910 msgr2=0x7fcfa80b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:13.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:13.977+0000 7fcfc457c700 1 --2- 172.21.15.32:0/3528817839 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fcfa80b2910 0x7fcfa80b4dd0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fcfb800b5c0 tx=0x7fcfb8005960 comp rx=0 tx=0).stop 2024-05-03T18:45:13.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:13.977+0000 7fcfc457c700 1 -- 172.21.15.32:0/3528817839 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcfbc06c3d0 msgr2=0x7fcfbc1029a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:13.978 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:13.977+0000 7fcfc457c700 1 --2- 172.21.15.32:0/3528817839 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcfbc06c3d0 0x7fcfbc1029a0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fcfac00d950 tx=0x7fcfac00dc60 comp rx=0 tx=0).stop 2024-05-03T18:45:13.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:13.977+0000 7fcfc457c700 1 -- 172.21.15.32:0/3528817839 shutdown_connections 2024-05-03T18:45:13.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:13.977+0000 7fcfc457c700 1 --2- 172.21.15.32:0/3528817839 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fcfa80b2910 0x7fcfa80b4dd0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:13.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:13.977+0000 7fcfc457c700 1 --2- 172.21.15.32:0/3528817839 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcfbc109500 0x7fcfbc102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:13.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:13.977+0000 7fcfc457c700 1 --2- 172.21.15.32:0/3528817839 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcfbc06ba10 0x7fcfbc102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:13.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:13.977+0000 7fcfc457c700 1 --2- 172.21.15.32:0/3528817839 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcfbc06c3d0 0x7fcfbc1029a0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:13.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:13.977+0000 7fcfc457c700 1 -- 172.21.15.32:0/3528817839 >> 172.21.15.32:0/3528817839 conn(0x7fcfbc079310 msgr2=0x7fcfbc1011f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:45:13.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:13.977+0000 7fcfc457c700 1 -- 172.21.15.32:0/3528817839 shutdown_connections 2024-05-03T18:45:13.979 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:13.977+0000 7fcfc457c700 1 -- 172.21.15.32:0/3528817839 wait complete. 2024-05-03T18:45:13.997 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:45:13 smithi045 ceph-mon[33299]: pgmap v291: 97 pgs: 97 active+clean; 582 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-05-03T18:45:13.997 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:45:13 smithi045 ceph-mon[33299]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-05-03T18:45:14.613 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid f500455e-097b-11ef-bc97-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-05-03T18:45:14.862 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:45:15.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:45:15 smithi032 ceph-mon[29763]: from='client.14712 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:45:15.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:45:15 smithi032 ceph-mon[29763]: pgmap v292: 97 pgs: 97 active+clean; 588 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 2.7 KiB/s wr, 12 op/s 2024-05-03T18:45:15.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:45:15 smithi113 ceph-mon[33540]: from='client.14712 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:45:15.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:45:15 smithi113 ceph-mon[33540]: pgmap v292: 97 pgs: 97 active+clean; 588 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 2.7 KiB/s wr, 12 op/s 2024-05-03T18:45:15.997 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:45:15 smithi045 ceph-mon[33299]: from='client.14712 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:45:15.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:45:15 smithi045 ceph-mon[33299]: pgmap v292: 97 pgs: 97 active+clean; 588 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 2.7 KiB/s wr, 12 op/s 2024-05-03T18:45:16.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:16.450+0000 7f4b811fb700 1 -- 172.21.15.32:0/1760621915 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4b7c06bc70 msgr2=0x7f4b7c06c0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:16.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:16.450+0000 7f4b811fb700 1 --2- 172.21.15.32:0/1760621915 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4b7c06bc70 0x7f4b7c06c0f0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f4b6c0099c0 tx=0x7f4b6c009cd0 comp rx=0 tx=0).stop 2024-05-03T18:45:16.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:16.450+0000 7f4b811fb700 1 -- 172.21.15.32:0/1760621915 shutdown_connections 2024-05-03T18:45:16.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:16.450+0000 7f4b811fb700 1 --2- 172.21.15.32:0/1760621915 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b7c06c630 0x7f4b7c110360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:16.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:16.450+0000 7f4b811fb700 1 --2- 172.21.15.32:0/1760621915 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4b7c109590 0x7f4b7c109970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:16.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:16.450+0000 7f4b811fb700 1 --2- 172.21.15.32:0/1760621915 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4b7c06bc70 0x7f4b7c06c0f0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:16.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:16.450+0000 7f4b811fb700 1 -- 172.21.15.32:0/1760621915 >> 172.21.15.32:0/1760621915 conn(0x7f4b7c079440 msgr2=0x7f4b7c079850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:45:16.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:16.450+0000 7f4b811fb700 1 -- 172.21.15.32:0/1760621915 shutdown_connections 2024-05-03T18:45:16.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:16.451+0000 7f4b811fb700 1 -- 172.21.15.32:0/1760621915 wait complete. 2024-05-03T18:45:16.453 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:16.451+0000 7f4b811fb700 1 Processor -- start 2024-05-03T18:45:16.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:16.451+0000 7f4b811fb700 1 -- start start 2024-05-03T18:45:16.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:16.452+0000 7f4b811fb700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b7c06bc70 0x7f4b7c19f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:16.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:16.452+0000 7f4b811fb700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4b7c06c630 0x7f4b7c19fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:16.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:16.452+0000 7f4b811fb700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4b7c109590 0x7f4b7c199a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:16.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:16.452+0000 7f4b811fb700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f4b7c113700 con 0x7f4b7c06c630 2024-05-03T18:45:16.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:16.452+0000 7f4b811fb700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4b7c113580 con 0x7f4b7c06bc70 2024-05-03T18:45:16.455 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:16.452+0000 7f4b811fb700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f4b7c113880 con 0x7f4b7c109590 2024-05-03T18:45:16.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:16.452+0000 7f4b7a59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4b7c06c630 0x7f4b7c19fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:16.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:16.452+0000 7f4b7a59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4b7c06c630 0x7f4b7c19fdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48670/0 (socket says 172.21.15.32:48670) 2024-05-03T18:45:16.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:16.452+0000 7f4b7a59c700 1 -- 172.21.15.32:0/3122872108 learned_addr learned my addr 172.21.15.32:0/3122872108 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:45:16.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:16.452+0000 7f4b7ad9d700 1 --2- 172.21.15.32:0/3122872108 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b7c06bc70 0x7f4b7c19f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:16.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:16.452+0000 7f4b7b59e700 1 --2- 172.21.15.32:0/3122872108 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4b7c109590 0x7f4b7c199a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:16.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:16.452+0000 7f4b7a59c700 1 -- 172.21.15.32:0/3122872108 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4b7c109590 msgr2=0x7f4b7c199a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:16.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:16.452+0000 7f4b7a59c700 1 --2- 172.21.15.32:0/3122872108 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4b7c109590 0x7f4b7c199a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:16.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:16.452+0000 7f4b7a59c700 1 -- 172.21.15.32:0/3122872108 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b7c06bc70 msgr2=0x7f4b7c19f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:16.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:16.452+0000 7f4b7a59c700 1 --2- 172.21.15.32:0/3122872108 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b7c06bc70 0x7f4b7c19f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:16.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:16.452+0000 7f4b7a59c700 1 -- 172.21.15.32:0/3122872108 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b7c19a1a0 con 0x7f4b7c06c630 2024-05-03T18:45:16.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:16.452+0000 7f4b7b59e700 1 --2- 172.21.15.32:0/3122872108 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4b7c109590 0x7f4b7c199a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T18:45:16.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:16.452+0000 7f4b7a59c700 1 --2- 172.21.15.32:0/3122872108 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4b7c06c630 0x7f4b7c19fdc0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f4b6c00b580 tx=0x7f4b6c0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:45:16.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:16.453+0000 7f4b63fff700 1 -- 172.21.15.32:0/3122872108 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b6c027070 con 0x7f4b7c06c630 2024-05-03T18:45:16.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:16.453+0000 7f4b811fb700 1 -- 172.21.15.32:0/3122872108 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b7c19a430 con 0x7f4b7c06c630 2024-05-03T18:45:16.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:16.453+0000 7f4b811fb700 1 -- 172.21.15.32:0/3122872108 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b7c1a6750 con 0x7f4b7c06c630 2024-05-03T18:45:16.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:16.453+0000 7f4b63fff700 1 -- 172.21.15.32:0/3122872108 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b6c003f20 con 0x7f4b7c06c630 2024-05-03T18:45:16.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:16.453+0000 7f4b63fff700 1 -- 172.21.15.32:0/3122872108 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b6c022d70 con 0x7f4b7c06c630 2024-05-03T18:45:16.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:16.454+0000 7f4b811fb700 1 -- 172.21.15.32:0/3122872108 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b68005330 con 0x7f4b7c06c630 2024-05-03T18:45:16.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:16.456+0000 7f4b63fff700 1 -- 172.21.15.32:0/3122872108 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f4b6c022470 con 0x7f4b7c06c630 2024-05-03T18:45:16.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:16.456+0000 7f4b63fff700 1 --2- 172.21.15.32:0/3122872108 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f4b640b2990 0x7f4b640b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:16.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:16.457+0000 7f4b7ad9d700 1 --2- 172.21.15.32:0/3122872108 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f4b640b2990 0x7f4b640b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:16.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:16.457+0000 7f4b63fff700 1 -- 172.21.15.32:0/3122872108 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7979+0+0 (secure 0 0 0) 0x7f4b6c0e3040 con 0x7f4b7c06c630 2024-05-03T18:45:16.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:16.457+0000 7f4b7ad9d700 1 --2- 172.21.15.32:0/3122872108 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f4b640b2990 0x7f4b640b4e50 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c19ab80 tx=0x7f4b70009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:45:16.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:16.460+0000 7f4b63fff700 1 -- 172.21.15.32:0/3122872108 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f4b6c013070 con 0x7f4b7c06c630 2024-05-03T18:45:16.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:16.713+0000 7f4b811fb700 1 -- 172.21.15.32:0/3122872108 --> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f4b68000bc0 con 0x7f4b640b2990 2024-05-03T18:45:16.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:16.720+0000 7f4b63fff700 1 -- 172.21.15.32:0/3122872108 <== mgr.14152 v2:172.21.15.32:6800/1796547904 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f4b68000bc0 con 0x7f4b640b2990 2024-05-03T18:45:16.721 INFO:teuthology.orchestra.run.smithi032.stdout:/volumes/qa/sv_0/9d8cba98-e83e-4b62-84e9-8cb25548d3d7 2024-05-03T18:45:16.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:16.722+0000 7f4b811fb700 1 -- 172.21.15.32:0/3122872108 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f4b640b2990 msgr2=0x7f4b640b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:16.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:16.722+0000 7f4b811fb700 1 --2- 172.21.15.32:0/3122872108 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f4b640b2990 0x7f4b640b4e50 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c19ab80 tx=0x7f4b70009190 comp rx=0 tx=0).stop 2024-05-03T18:45:16.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:16.722+0000 7f4b811fb700 1 -- 172.21.15.32:0/3122872108 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4b7c06c630 msgr2=0x7f4b7c19fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:16.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:16.722+0000 7f4b811fb700 1 --2- 172.21.15.32:0/3122872108 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4b7c06c630 0x7f4b7c19fdc0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f4b6c00b580 tx=0x7f4b6c0047b0 comp rx=0 tx=0).stop 2024-05-03T18:45:16.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:16.722+0000 7f4b811fb700 1 -- 172.21.15.32:0/3122872108 shutdown_connections 2024-05-03T18:45:16.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:16.722+0000 7f4b811fb700 1 --2- 172.21.15.32:0/3122872108 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f4b640b2990 0x7f4b640b4e50 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:16.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:16.722+0000 7f4b811fb700 1 --2- 172.21.15.32:0/3122872108 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4b7c109590 0x7f4b7c199a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:16.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:16.722+0000 7f4b811fb700 1 --2- 172.21.15.32:0/3122872108 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4b7c06bc70 0x7f4b7c19f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:16.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:16.722+0000 7f4b811fb700 1 --2- 172.21.15.32:0/3122872108 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4b7c06c630 0x7f4b7c19fdc0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:16.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:16.722+0000 7f4b811fb700 1 -- 172.21.15.32:0/3122872108 >> 172.21.15.32:0/3122872108 conn(0x7f4b7c079440 msgr2=0x7f4b7c100fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:45:16.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:16.723+0000 7f4b811fb700 1 -- 172.21.15.32:0/3122872108 shutdown_connections 2024-05-03T18:45:16.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:16.723+0000 7f4b811fb700 1 -- 172.21.15.32:0/3122872108 wait complete. 2024-05-03T18:45:17.454 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid f500455e-097b-11ef-bc97-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-05-03T18:45:17.704 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:45:17.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:45:17 smithi113 ceph-mon[33540]: pgmap v293: 97 pgs: 97 active+clean; 588 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 2.2 KiB/s wr, 11 op/s 2024-05-03T18:45:17.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:45:17 smithi032 ceph-mon[29763]: pgmap v293: 97 pgs: 97 active+clean; 588 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 2.2 KiB/s wr, 11 op/s 2024-05-03T18:45:17.997 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:45:17 smithi045 ceph-mon[33299]: pgmap v293: 97 pgs: 97 active+clean; 588 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 2.2 KiB/s wr, 11 op/s 2024-05-03T18:45:18.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:45:18 smithi113 ceph-mon[33540]: from='client.14718 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:45:18.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:45:18 smithi032 ceph-mon[29763]: from='client.14718 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:45:18.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:45:18 smithi045 ceph-mon[33299]: from='client.14718 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:45:19.372 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:19.369+0000 7efc00725700 1 -- 172.21.15.32:0/3571404335 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efbf8108410 msgr2=0x7efbf81087f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:19.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:19.369+0000 7efc00725700 1 --2- 172.21.15.32:0/3571404335 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efbf8108410 0x7efbf81087f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7efbf400b0a0 tx=0x7efbf400b3b0 comp rx=0 tx=0).stop 2024-05-03T18:45:19.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:19.369+0000 7efc00725700 1 -- 172.21.15.32:0/3571404335 shutdown_connections 2024-05-03T18:45:19.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:19.369+0000 7efc00725700 1 --2- 172.21.15.32:0/3571404335 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efbf8102d60 0x7efbf810f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:19.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:19.369+0000 7efc00725700 1 --2- 172.21.15.32:0/3571404335 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efbf8108410 0x7efbf81087f0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:19.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:19.369+0000 7efc00725700 1 --2- 172.21.15.32:0/3571404335 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efbf81023a0 0x7efbf8102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:19.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:19.369+0000 7efc00725700 1 -- 172.21.15.32:0/3571404335 >> 172.21.15.32:0/3571404335 conn(0x7efbf8079350 msgr2=0x7efbf8079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:45:19.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:19.369+0000 7efc00725700 1 -- 172.21.15.32:0/3571404335 shutdown_connections 2024-05-03T18:45:19.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:19.369+0000 7efc00725700 1 -- 172.21.15.32:0/3571404335 wait complete. 2024-05-03T18:45:19.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:19.370+0000 7efc00725700 1 Processor -- start 2024-05-03T18:45:19.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:19.370+0000 7efc00725700 1 -- start start 2024-05-03T18:45:19.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:19.370+0000 7efc00725700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efbf81023a0 0x7efbf819e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:19.373 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:19.370+0000 7efc00725700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efbf8102d60 0x7efbf819eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:19.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:19.370+0000 7efc00725700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efbf8108410 0x7efbf8198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:19.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:19.370+0000 7efc00725700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7efbf8112560 con 0x7efbf8102d60 2024-05-03T18:45:19.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:19.370+0000 7efc00725700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7efbf81123e0 con 0x7efbf81023a0 2024-05-03T18:45:19.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:19.370+0000 7efc00725700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7efbf81126e0 con 0x7efbf8108410 2024-05-03T18:45:19.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:19.370+0000 7efbfdcc0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efbf8102d60 0x7efbf819eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:19.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:19.370+0000 7efbfdcc0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efbf8102d60 0x7efbf819eca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48690/0 (socket says 172.21.15.32:48690) 2024-05-03T18:45:19.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:19.370+0000 7efbfdcc0700 1 -- 172.21.15.32:0/1557361032 learned_addr learned my addr 172.21.15.32:0/1557361032 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:45:19.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:19.370+0000 7efbfecc2700 1 --2- 172.21.15.32:0/1557361032 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efbf8108410 0x7efbf8198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:19.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:19.370+0000 7efbfe4c1700 1 --2- 172.21.15.32:0/1557361032 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efbf81023a0 0x7efbf819e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:19.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:19.371+0000 7efbfdcc0700 1 -- 172.21.15.32:0/1557361032 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efbf8108410 msgr2=0x7efbf8198920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:19.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:19.371+0000 7efbfdcc0700 1 --2- 172.21.15.32:0/1557361032 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efbf8108410 0x7efbf8198920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:19.374 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:19.371+0000 7efbfdcc0700 1 -- 172.21.15.32:0/1557361032 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efbf81023a0 msgr2=0x7efbf819e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:19.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:19.371+0000 7efbfdcc0700 1 --2- 172.21.15.32:0/1557361032 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efbf81023a0 0x7efbf819e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:19.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:19.371+0000 7efbfdcc0700 1 -- 172.21.15.32:0/1557361032 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efbf8199080 con 0x7efbf8102d60 2024-05-03T18:45:19.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:19.371+0000 7efbfdcc0700 1 --2- 172.21.15.32:0/1557361032 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efbf8102d60 0x7efbf819eca0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7efbe800d980 tx=0x7efbe800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:45:19.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:19.371+0000 7efbef7fe700 1 -- 172.21.15.32:0/1557361032 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbe8017070 con 0x7efbf8102d60 2024-05-03T18:45:19.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:19.371+0000 7efbef7fe700 1 -- 172.21.15.32:0/1557361032 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efbe8005020 con 0x7efbf8102d60 2024-05-03T18:45:19.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:19.371+0000 7efbef7fe700 1 -- 172.21.15.32:0/1557361032 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbe800fa10 con 0x7efbf8102d60 2024-05-03T18:45:19.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:19.371+0000 7efc00725700 1 -- 172.21.15.32:0/1557361032 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efbf8199370 con 0x7efbf8102d60 2024-05-03T18:45:19.375 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:19.371+0000 7efc00725700 1 -- 172.21.15.32:0/1557361032 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efbf81a5630 con 0x7efbf8102d60 2024-05-03T18:45:19.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:19.372+0000 7efc00725700 1 -- 172.21.15.32:0/1557361032 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efbf806a190 con 0x7efbf8102d60 2024-05-03T18:45:19.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:19.373+0000 7efbef7fe700 1 -- 172.21.15.32:0/1557361032 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7efbe8004a30 con 0x7efbf8102d60 2024-05-03T18:45:19.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:19.374+0000 7efbef7fe700 1 --2- 172.21.15.32:0/1557361032 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7efbe40b2910 0x7efbe40b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:19.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:19.374+0000 7efbfe4c1700 1 --2- 172.21.15.32:0/1557361032 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7efbe40b2910 0x7efbe40b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:19.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:19.374+0000 7efbef7fe700 1 -- 172.21.15.32:0/1557361032 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7979+0+0 (secure 0 0 0) 0x7efbe80e24c0 con 0x7efbf8102d60 2024-05-03T18:45:19.376 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:19.375+0000 7efbfe4c1700 1 --2- 172.21.15.32:0/1557361032 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7efbe40b2910 0x7efbe40b4dd0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7efbf4009e60 tx=0x7efbf400b7b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:45:19.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:19.378+0000 7efbef7fe700 1 -- 172.21.15.32:0/1557361032 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7efbe8020020 con 0x7efbf8102d60 2024-05-03T18:45:19.626 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:19.625+0000 7efc00725700 1 -- 172.21.15.32:0/1557361032 --> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7efbf80638e0 con 0x7efbe40b2910 2024-05-03T18:45:19.654 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:19.653+0000 7efbef7fe700 1 -- 172.21.15.32:0/1557361032 <== mgr.14152 v2:172.21.15.32:6800/1796547904 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7efbf80638e0 con 0x7efbe40b2910 2024-05-03T18:45:19.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:19.655+0000 7efc00725700 1 -- 172.21.15.32:0/1557361032 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7efbe40b2910 msgr2=0x7efbe40b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:19.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:19.655+0000 7efc00725700 1 --2- 172.21.15.32:0/1557361032 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7efbe40b2910 0x7efbe40b4dd0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7efbf4009e60 tx=0x7efbf400b7b0 comp rx=0 tx=0).stop 2024-05-03T18:45:19.656 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:19.655+0000 7efc00725700 1 -- 172.21.15.32:0/1557361032 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efbf8102d60 msgr2=0x7efbf819eca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:19.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:19.655+0000 7efc00725700 1 --2- 172.21.15.32:0/1557361032 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efbf8102d60 0x7efbf819eca0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7efbe800d980 tx=0x7efbe800dc90 comp rx=0 tx=0).stop 2024-05-03T18:45:19.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:19.655+0000 7efc00725700 1 -- 172.21.15.32:0/1557361032 shutdown_connections 2024-05-03T18:45:19.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:19.655+0000 7efc00725700 1 --2- 172.21.15.32:0/1557361032 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7efbe40b2910 0x7efbe40b4dd0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:19.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:19.655+0000 7efc00725700 1 --2- 172.21.15.32:0/1557361032 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efbf8108410 0x7efbf8198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:19.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:19.655+0000 7efc00725700 1 --2- 172.21.15.32:0/1557361032 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efbf81023a0 0x7efbf819e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:19.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:19.655+0000 7efc00725700 1 --2- 172.21.15.32:0/1557361032 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efbf8102d60 0x7efbf819eca0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:19.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:19.655+0000 7efc00725700 1 -- 172.21.15.32:0/1557361032 >> 172.21.15.32:0/1557361032 conn(0x7efbf8079350 msgr2=0x7efbf8101160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:45:19.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:19.655+0000 7efc00725700 1 -- 172.21.15.32:0/1557361032 shutdown_connections 2024-05-03T18:45:19.657 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:19.655+0000 7efc00725700 1 -- 172.21.15.32:0/1557361032 wait complete. 2024-05-03T18:45:19.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:45:19 smithi113 ceph-mon[33540]: pgmap v294: 97 pgs: 97 active+clean; 602 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 2.5 KiB/s wr, 10 op/s 2024-05-03T18:45:19.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:45:19 smithi032 ceph-mon[29763]: pgmap v294: 97 pgs: 97 active+clean; 602 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 2.5 KiB/s wr, 10 op/s 2024-05-03T18:45:19.997 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:45:19 smithi045 ceph-mon[33299]: pgmap v294: 97 pgs: 97 active+clean; 602 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 2.5 KiB/s wr, 10 op/s 2024-05-03T18:45:20.448 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid f500455e-097b-11ef-bc97-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-05-03T18:45:20.705 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:45:21.744 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:45:21 smithi032 ceph-mon[29763]: from='client.14724 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:45:21.744 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:45:21 smithi032 ceph-mon[29763]: pgmap v295: 97 pgs: 97 active+clean; 602 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-05-03T18:45:21.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:45:21 smithi113 ceph-mon[33540]: from='client.14724 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:45:21.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:45:21 smithi113 ceph-mon[33540]: pgmap v295: 97 pgs: 97 active+clean; 602 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-05-03T18:45:21.997 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:45:21 smithi045 ceph-mon[33299]: from='client.14724 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:45:21.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:45:21 smithi045 ceph-mon[33299]: pgmap v295: 97 pgs: 97 active+clean; 602 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-05-03T18:45:22.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:22.230+0000 7ff603442700 1 -- 172.21.15.32:0/3276935178 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff5fc109590 msgr2=0x7ff5fc109970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:22.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:22.230+0000 7ff603442700 1 --2- 172.21.15.32:0/3276935178 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff5fc109590 0x7ff5fc109970 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ff5f80099c0 tx=0x7ff5f8009cd0 comp rx=0 tx=0).stop 2024-05-03T18:45:22.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:22.230+0000 7ff603442700 1 -- 172.21.15.32:0/3276935178 shutdown_connections 2024-05-03T18:45:22.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:22.230+0000 7ff603442700 1 --2- 172.21.15.32:0/3276935178 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff5fc06c630 0x7ff5fc110360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:22.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:22.230+0000 7ff603442700 1 --2- 172.21.15.32:0/3276935178 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff5fc109590 0x7ff5fc109970 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:22.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:22.230+0000 7ff603442700 1 --2- 172.21.15.32:0/3276935178 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff5fc06bc70 0x7ff5fc06c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:22.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:22.230+0000 7ff603442700 1 -- 172.21.15.32:0/3276935178 >> 172.21.15.32:0/3276935178 conn(0x7ff5fc079440 msgr2=0x7ff5fc079850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:45:22.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:22.231+0000 7ff603442700 1 -- 172.21.15.32:0/3276935178 shutdown_connections 2024-05-03T18:45:22.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:22.231+0000 7ff603442700 1 -- 172.21.15.32:0/3276935178 wait complete. 2024-05-03T18:45:22.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:22.231+0000 7ff603442700 1 Processor -- start 2024-05-03T18:45:22.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:22.231+0000 7ff603442700 1 -- start start 2024-05-03T18:45:22.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:22.231+0000 7ff603442700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff5fc06bc70 0x7ff5fc102410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:22.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:22.231+0000 7ff603442700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff5fc06c630 0x7ff5fc102950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:22.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:22.231+0000 7ff603442700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff5fc109590 0x7ff5fc102e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:22.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:22.231+0000 7ff603442700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff5fc113750 con 0x7ff5fc109590 2024-05-03T18:45:22.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:22.231+0000 7ff603442700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff5fc1135d0 con 0x7ff5fc06bc70 2024-05-03T18:45:22.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:22.231+0000 7ff603442700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7ff5fc1138d0 con 0x7ff5fc06c630 2024-05-03T18:45:22.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:22.232+0000 7ff6019df700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff5fc109590 0x7ff5fc102e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:22.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:22.232+0000 7ff6019df700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff5fc109590 0x7ff5fc102e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48706/0 (socket says 172.21.15.32:48706) 2024-05-03T18:45:22.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:22.232+0000 7ff6019df700 1 -- 172.21.15.32:0/2018819287 learned_addr learned my addr 172.21.15.32:0/2018819287 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:45:22.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:22.232+0000 7ff6009dd700 1 --2- 172.21.15.32:0/2018819287 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff5fc06c630 0x7ff5fc102950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:22.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:22.232+0000 7ff6011de700 1 --2- 172.21.15.32:0/2018819287 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff5fc06bc70 0x7ff5fc102410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:22.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:22.232+0000 7ff6019df700 1 -- 172.21.15.32:0/2018819287 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff5fc06c630 msgr2=0x7ff5fc102950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:22.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:22.232+0000 7ff6019df700 1 --2- 172.21.15.32:0/2018819287 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff5fc06c630 0x7ff5fc102950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:22.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:22.232+0000 7ff6019df700 1 -- 172.21.15.32:0/2018819287 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff5fc06bc70 msgr2=0x7ff5fc102410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:22.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:22.232+0000 7ff6019df700 1 --2- 172.21.15.32:0/2018819287 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff5fc06bc70 0x7ff5fc102410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:22.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:22.232+0000 7ff6019df700 1 -- 172.21.15.32:0/2018819287 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5fc1a8110 con 0x7ff5fc109590 2024-05-03T18:45:22.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:22.232+0000 7ff6011de700 1 --2- 172.21.15.32:0/2018819287 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff5fc06bc70 0x7ff5fc102410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T18:45:22.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:22.232+0000 7ff6019df700 1 --2- 172.21.15.32:0/2018819287 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff5fc109590 0x7ff5fc102e90 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ff5f400b970 tx=0x7ff5f400bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:45:22.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:22.232+0000 7ff5f27fc700 1 -- 172.21.15.32:0/2018819287 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5f4019070 con 0x7ff5fc109590 2024-05-03T18:45:22.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:22.232+0000 7ff603442700 1 -- 172.21.15.32:0/2018819287 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5fc1a8400 con 0x7ff5fc109590 2024-05-03T18:45:22.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:22.232+0000 7ff5f27fc700 1 -- 172.21.15.32:0/2018819287 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff5f400cde0 con 0x7ff5fc109590 2024-05-03T18:45:22.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:22.232+0000 7ff5f27fc700 1 -- 172.21.15.32:0/2018819287 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5f4007370 con 0x7ff5fc109590 2024-05-03T18:45:22.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:22.232+0000 7ff603442700 1 -- 172.21.15.32:0/2018819287 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5fc1a8940 con 0x7ff5fc109590 2024-05-03T18:45:22.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:22.234+0000 7ff603442700 1 -- 172.21.15.32:0/2018819287 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5fc10ba20 con 0x7ff5fc109590 2024-05-03T18:45:22.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:22.235+0000 7ff5f27fc700 1 -- 172.21.15.32:0/2018819287 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7ff5f401c070 con 0x7ff5fc109590 2024-05-03T18:45:22.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:22.235+0000 7ff5f27fc700 1 --2- 172.21.15.32:0/2018819287 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7ff5e80b2910 0x7ff5e80b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:22.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:22.235+0000 7ff6011de700 1 --2- 172.21.15.32:0/2018819287 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7ff5e80b2910 0x7ff5e80b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:22.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:22.236+0000 7ff5f27fc700 1 -- 172.21.15.32:0/2018819287 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7979+0+0 (secure 0 0 0) 0x7ff5f40e1450 con 0x7ff5fc109590 2024-05-03T18:45:22.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:22.236+0000 7ff6011de700 1 --2- 172.21.15.32:0/2018819287 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7ff5e80b2910 0x7ff5e80b4dd0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ff5f8009fd0 tx=0x7ff5f8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:45:22.241 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:22.240+0000 7ff5f27fc700 1 -- 172.21.15.32:0/2018819287 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7ff5f40a92d0 con 0x7ff5fc109590 2024-05-03T18:45:22.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:22.493+0000 7ff603442700 1 -- 172.21.15.32:0/2018819287 --> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7ff5fc0638e0 con 0x7ff5e80b2910 2024-05-03T18:45:22.499 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:22.498+0000 7ff5f27fc700 1 -- 172.21.15.32:0/2018819287 <== mgr.14152 v2:172.21.15.32:6800/1796547904 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7ff5fc0638e0 con 0x7ff5e80b2910 2024-05-03T18:45:22.499 INFO:teuthology.orchestra.run.smithi032.stdout:/volumes/qa/sv_1/eb69fd66-9cd9-414a-89e3-85fc2655cfa5 2024-05-03T18:45:22.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:22.500+0000 7ff603442700 1 -- 172.21.15.32:0/2018819287 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7ff5e80b2910 msgr2=0x7ff5e80b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:22.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:22.500+0000 7ff603442700 1 --2- 172.21.15.32:0/2018819287 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7ff5e80b2910 0x7ff5e80b4dd0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ff5f8009fd0 tx=0x7ff5f8005fb0 comp rx=0 tx=0).stop 2024-05-03T18:45:22.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:22.501+0000 7ff603442700 1 -- 172.21.15.32:0/2018819287 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff5fc109590 msgr2=0x7ff5fc102e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:22.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:22.501+0000 7ff603442700 1 --2- 172.21.15.32:0/2018819287 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff5fc109590 0x7ff5fc102e90 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ff5f400b970 tx=0x7ff5f400bc80 comp rx=0 tx=0).stop 2024-05-03T18:45:22.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:22.501+0000 7ff603442700 1 -- 172.21.15.32:0/2018819287 shutdown_connections 2024-05-03T18:45:22.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:22.501+0000 7ff603442700 1 --2- 172.21.15.32:0/2018819287 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7ff5e80b2910 0x7ff5e80b4dd0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:22.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:22.501+0000 7ff603442700 1 --2- 172.21.15.32:0/2018819287 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff5fc109590 0x7ff5fc102e90 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:22.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:22.501+0000 7ff603442700 1 --2- 172.21.15.32:0/2018819287 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff5fc06bc70 0x7ff5fc102410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:22.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:22.501+0000 7ff603442700 1 --2- 172.21.15.32:0/2018819287 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff5fc06c630 0x7ff5fc102950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:22.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:22.501+0000 7ff603442700 1 -- 172.21.15.32:0/2018819287 >> 172.21.15.32:0/2018819287 conn(0x7ff5fc079440 msgr2=0x7ff5fc100e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:45:22.502 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:22.501+0000 7ff603442700 1 -- 172.21.15.32:0/2018819287 shutdown_connections 2024-05-03T18:45:22.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:22.501+0000 7ff603442700 1 -- 172.21.15.32:0/2018819287 wait complete. 2024-05-03T18:45:23.328 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid f500455e-097b-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-03T18:45:23.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:45:23 smithi032 ceph-mon[29763]: pgmap v296: 97 pgs: 97 active+clean; 602 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.5 KiB/s wr, 7 op/s 2024-05-03T18:45:23.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:45:23 smithi032 ceph-mon[29763]: from='client.14730 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:45:23.584 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:45:23.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:45:23 smithi113 ceph-mon[33540]: pgmap v296: 97 pgs: 97 active+clean; 602 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.5 KiB/s wr, 7 op/s 2024-05-03T18:45:23.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:45:23 smithi113 ceph-mon[33540]: from='client.14730 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:45:23.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:45:23 smithi045 ceph-mon[33299]: pgmap v296: 97 pgs: 97 active+clean; 602 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.5 KiB/s wr, 7 op/s 2024-05-03T18:45:23.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:45:23 smithi045 ceph-mon[33299]: from='client.14730 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:45:25.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:25.302+0000 7f7b26547700 1 -- 172.21.15.32:0/4161812024 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7b20102d60 msgr2=0x7f7b2010f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:25.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:25.302+0000 7f7b26547700 1 --2- 172.21.15.32:0/4161812024 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7b20102d60 0x7f7b2010f250 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f7b1800b0a0 tx=0x7f7b1800b3b0 comp rx=0 tx=0).stop 2024-05-03T18:45:25.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:25.303+0000 7f7b26547700 1 -- 172.21.15.32:0/4161812024 shutdown_connections 2024-05-03T18:45:25.304 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:25.303+0000 7f7b26547700 1 --2- 172.21.15.32:0/4161812024 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7b20102d60 0x7f7b2010f250 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:25.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:25.303+0000 7f7b26547700 1 --2- 172.21.15.32:0/4161812024 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7b20108410 0x7f7b201087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:25.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:25.303+0000 7f7b26547700 1 --2- 172.21.15.32:0/4161812024 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7b201023a0 0x7f7b20102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:25.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:25.303+0000 7f7b26547700 1 -- 172.21.15.32:0/4161812024 >> 172.21.15.32:0/4161812024 conn(0x7f7b20079350 msgr2=0x7f7b20079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:45:25.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:25.303+0000 7f7b26547700 1 -- 172.21.15.32:0/4161812024 shutdown_connections 2024-05-03T18:45:25.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:25.303+0000 7f7b26547700 1 -- 172.21.15.32:0/4161812024 wait complete. 2024-05-03T18:45:25.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:25.303+0000 7f7b26547700 1 Processor -- start 2024-05-03T18:45:25.305 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:25.303+0000 7f7b26547700 1 -- start start 2024-05-03T18:45:25.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:25.304+0000 7f7b26547700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7b201023a0 0x7f7b2019e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:25.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:25.304+0000 7f7b26547700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7b20102d60 0x7f7b2019ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:25.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:25.304+0000 7f7b26547700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7b20108410 0x7f7b20198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:25.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:25.304+0000 7f7b26547700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f7b20112630 con 0x7f7b20102d60 2024-05-03T18:45:25.307 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:25.304+0000 7f7b26547700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7b201124b0 con 0x7f7b20108410 2024-05-03T18:45:25.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:25.304+0000 7f7b26547700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f7b201127b0 con 0x7f7b201023a0 2024-05-03T18:45:25.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:25.304+0000 7f7b1f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7b20102d60 0x7f7b2019ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:25.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:25.304+0000 7f7b1f7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7b20102d60 0x7f7b2019ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42856/0 (socket says 172.21.15.32:42856) 2024-05-03T18:45:25.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:25.304+0000 7f7b1f7fe700 1 -- 172.21.15.32:0/562687233 learned_addr learned my addr 172.21.15.32:0/562687233 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:45:25.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:25.304+0000 7f7b1ffff700 1 --2- 172.21.15.32:0/562687233 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7b201023a0 0x7f7b2019e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:25.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:25.304+0000 7f7b24ae4700 1 --2- 172.21.15.32:0/562687233 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7b20108410 0x7f7b20198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:25.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:25.304+0000 7f7b1f7fe700 1 -- 172.21.15.32:0/562687233 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7b201023a0 msgr2=0x7f7b2019e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:25.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:25.304+0000 7f7b1f7fe700 1 --2- 172.21.15.32:0/562687233 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7b201023a0 0x7f7b2019e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:25.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:25.304+0000 7f7b1f7fe700 1 -- 172.21.15.32:0/562687233 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7b20108410 msgr2=0x7f7b20198a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:25.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:25.304+0000 7f7b1f7fe700 1 --2- 172.21.15.32:0/562687233 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7b20108410 0x7f7b20198a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:25.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:25.304+0000 7f7b1f7fe700 1 -- 172.21.15.32:0/562687233 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b20199170 con 0x7f7b20102d60 2024-05-03T18:45:25.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:25.304+0000 7f7b24ae4700 1 --2- 172.21.15.32:0/562687233 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7b20108410 0x7f7b20198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T18:45:25.308 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:25.305+0000 7f7b1f7fe700 1 --2- 172.21.15.32:0/562687233 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7b20102d60 0x7f7b2019ed90 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f7b1400d9e0 tx=0x7f7b1400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:45:25.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:25.305+0000 7f7b1d7fa700 1 -- 172.21.15.32:0/562687233 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b1401b070 con 0x7f7b20102d60 2024-05-03T18:45:25.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:25.305+0000 7f7b1d7fa700 1 -- 172.21.15.32:0/562687233 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7b1400ec50 con 0x7f7b20102d60 2024-05-03T18:45:25.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:25.305+0000 7f7b26547700 1 -- 172.21.15.32:0/562687233 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b20199460 con 0x7f7b20102d60 2024-05-03T18:45:25.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:25.305+0000 7f7b1d7fa700 1 -- 172.21.15.32:0/562687233 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b14014630 con 0x7f7b20102d60 2024-05-03T18:45:25.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:25.305+0000 7f7b26547700 1 -- 172.21.15.32:0/562687233 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b2006b180 con 0x7f7b20102d60 2024-05-03T18:45:25.309 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:25.306+0000 7f7b26547700 1 -- 172.21.15.32:0/562687233 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b2006a190 con 0x7f7b20102d60 2024-05-03T18:45:25.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:25.308+0000 7f7b1d7fa700 1 -- 172.21.15.32:0/562687233 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f7b1401e070 con 0x7f7b20102d60 2024-05-03T18:45:25.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:25.309+0000 7f7b1d7fa700 1 --2- 172.21.15.32:0/562687233 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f7b080b2910 0x7f7b080b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:25.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:25.309+0000 7f7b1ffff700 1 --2- 172.21.15.32:0/562687233 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f7b080b2910 0x7f7b080b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:25.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:25.309+0000 7f7b1d7fa700 1 -- 172.21.15.32:0/562687233 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7979+0+0 (secure 0 0 0) 0x7f7b140e1b60 con 0x7f7b20102d60 2024-05-03T18:45:25.310 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:25.309+0000 7f7b1ffff700 1 --2- 172.21.15.32:0/562687233 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f7b080b2910 0x7f7b080b4dd0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f7b20199b50 tx=0x7f7b10009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:45:25.312 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:25.312+0000 7f7b1d7fa700 1 -- 172.21.15.32:0/562687233 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f7b140a98e0 con 0x7f7b20102d60 2024-05-03T18:45:25.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:45:25 smithi032 ceph-mon[29763]: pgmap v297: 97 pgs: 97 active+clean; 622 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 4.2 KiB/s wr, 9 op/s 2024-05-03T18:45:25.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:25.574+0000 7f7b26547700 1 -- 172.21.15.32:0/562687233 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7b20199f40 con 0x7f7b20102d60 2024-05-03T18:45:25.575 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:25.575+0000 7f7b1d7fa700 1 -- 172.21.15.32:0/562687233 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11862 (secure 0 0 0) 0x7f7b20199f40 con 0x7f7b20102d60 2024-05-03T18:45:25.576 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-03T18:45:25.576 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/535599929","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":535599929},{"type":"v1","addr":"172.21.15.32:6835","nonce":535599929}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/3101679868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":3101679868},{"type":"v1","addr":"172.21.15.32:6837","nonce":3101679868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14610,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/3838653921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":3838653921},{"type":"v1","addr":"172.21.15.32:6839","nonce":3838653921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14622,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6841/3019648436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6840","nonce":3019648436},{"type":"v1","addr":"172.21.15.32:6841","nonce":3019648436}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24380,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/1629869435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":1629869435},{"type":"v1","addr":"172.21.15.45:6833","nonce":1629869435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24398,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2997291166","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2997291166},{"type":"v1","addr":"172.21.15.45:6835","nonce":2997291166}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-03T18:44:51.155050+0000","modified":"2024-05-03T18:45:08.404530+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24478,"mds_1":24469,"mds_2":24463},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24425":{"gid":24425,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.45:6837/428055392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":428055392},{"type":"v1","addr":"172.21.15.45:6837","nonce":428055392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.113:6833/3323558791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":3323558791},{"type":"v1","addr":"172.21.15.113:6833","nonce":3323558791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"b","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.45:6839/970804040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":970804040},{"type":"v1","addr":"172.21.15.45:6839","nonce":970804040}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.113:6835/2588738276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2588738276},{"type":"v1","addr":"172.21.15.113:6835","nonce":2588738276}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.113:6837/3677284460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":3677284460},{"type":"v1","addr":"172.21.15.113:6837","nonce":3677284460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.113:6839/942050693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6838","nonce":942050693},{"type":"v1","addr":"172.21.15.113:6839","nonce":942050693}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24478,"qdb_cluster":[24469,24478,24463]},"id":1}]} 2024-05-03T18:45:25.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:25.577+0000 7f7b26547700 1 -- 172.21.15.32:0/562687233 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f7b080b2910 msgr2=0x7f7b080b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:25.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:25.577+0000 7f7b26547700 1 --2- 172.21.15.32:0/562687233 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f7b080b2910 0x7f7b080b4dd0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f7b20199b50 tx=0x7f7b10009190 comp rx=0 tx=0).stop 2024-05-03T18:45:25.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:25.577+0000 7f7b26547700 1 -- 172.21.15.32:0/562687233 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7b20102d60 msgr2=0x7f7b2019ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:25.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:25.577+0000 7f7b26547700 1 --2- 172.21.15.32:0/562687233 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7b20102d60 0x7f7b2019ed90 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f7b1400d9e0 tx=0x7f7b1400dcf0 comp rx=0 tx=0).stop 2024-05-03T18:45:25.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:25.577+0000 7f7b26547700 1 -- 172.21.15.32:0/562687233 shutdown_connections 2024-05-03T18:45:25.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:25.577+0000 7f7b26547700 1 --2- 172.21.15.32:0/562687233 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f7b080b2910 0x7f7b080b4dd0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:25.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:25.577+0000 7f7b26547700 1 --2- 172.21.15.32:0/562687233 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7b20108410 0x7f7b20198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:25.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:25.577+0000 7f7b26547700 1 --2- 172.21.15.32:0/562687233 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7b201023a0 0x7f7b2019e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:25.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:25.577+0000 7f7b26547700 1 --2- 172.21.15.32:0/562687233 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7b20102d60 0x7f7b2019ed90 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:25.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:25.577+0000 7f7b26547700 1 -- 172.21.15.32:0/562687233 >> 172.21.15.32:0/562687233 conn(0x7f7b20079350 msgr2=0x7f7b20101080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:45:25.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:25.577+0000 7f7b26547700 1 -- 172.21.15.32:0/562687233 shutdown_connections 2024-05-03T18:45:25.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:25.577+0000 7f7b26547700 1 -- 172.21.15.32:0/562687233 wait complete. 2024-05-03T18:45:25.579 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 25 2024-05-03T18:45:25.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:45:25 smithi113 ceph-mon[33540]: pgmap v297: 97 pgs: 97 active+clean; 622 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 4.2 KiB/s wr, 9 op/s 2024-05-03T18:45:25.997 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:45:25 smithi045 ceph-mon[33299]: pgmap v297: 97 pgs: 97 active+clean; 622 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 4.2 KiB/s wr, 9 op/s 2024-05-03T18:45:26.299 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid f500455e-097b-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-03T18:45:26.558 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:45:26.777 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:45:26 smithi032 ceph-mon[29763]: from='client.? 172.21.15.32:0/562687233' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:45:26.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:45:26 smithi113 ceph-mon[33540]: from='client.? 172.21.15.32:0/562687233' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:45:26.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:45:26 smithi045 ceph-mon[33299]: from='client.? 172.21.15.32:0/562687233' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:45:27.618 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:45:27 smithi032 ceph-mon[29763]: pgmap v298: 97 pgs: 97 active+clean; 622 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 3.5 KiB/s wr, 8 op/s 2024-05-03T18:45:27.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:45:27 smithi113 ceph-mon[33540]: pgmap v298: 97 pgs: 97 active+clean; 622 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 3.5 KiB/s wr, 8 op/s 2024-05-03T18:45:27.899 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:45:27 smithi045 ceph-mon[33299]: pgmap v298: 97 pgs: 97 active+clean; 622 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 3.5 KiB/s wr, 8 op/s 2024-05-03T18:45:28.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:28.091+0000 7f05c0392700 1 -- 172.21.15.32:0/3033196635 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05b806c590 msgr2=0x7f05b8110330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:28.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:28.091+0000 7f05c0392700 1 --2- 172.21.15.32:0/3033196635 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05b806c590 0x7f05b8110330 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f05b000b0a0 tx=0x7f05b000b3b0 comp rx=0 tx=0).stop 2024-05-03T18:45:28.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:28.091+0000 7f05c0392700 1 -- 172.21.15.32:0/3033196635 shutdown_connections 2024-05-03T18:45:28.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:28.091+0000 7f05c0392700 1 --2- 172.21.15.32:0/3033196635 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05b806c590 0x7f05b8110330 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:28.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:28.091+0000 7f05c0392700 1 --2- 172.21.15.32:0/3033196635 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05b8109510 0x7f05b81098f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:28.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:28.091+0000 7f05c0392700 1 --2- 172.21.15.32:0/3033196635 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f05b806bbd0 0x7f05b806c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:28.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:28.091+0000 7f05c0392700 1 -- 172.21.15.32:0/3033196635 >> 172.21.15.32:0/3033196635 conn(0x7f05b8079350 msgr2=0x7f05b8079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:45:28.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:28.091+0000 7f05c0392700 1 -- 172.21.15.32:0/3033196635 shutdown_connections 2024-05-03T18:45:28.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:28.091+0000 7f05c0392700 1 -- 172.21.15.32:0/3033196635 wait complete. 2024-05-03T18:45:28.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:28.092+0000 7f05c0392700 1 Processor -- start 2024-05-03T18:45:28.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:28.092+0000 7f05c0392700 1 -- start start 2024-05-03T18:45:28.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:28.092+0000 7f05c0392700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05b806bbd0 0x7f05b819f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:28.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:28.092+0000 7f05c0392700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f05b806c590 0x7f05b819fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:28.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:28.092+0000 7f05c0392700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05b8109510 0x7f05b8199900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:28.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:28.092+0000 7f05c0392700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f05b8113640 con 0x7f05b806bbd0 2024-05-03T18:45:28.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:28.092+0000 7f05c0392700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f05b81134c0 con 0x7f05b8109510 2024-05-03T18:45:28.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:28.092+0000 7f05c0392700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f05b81137c0 con 0x7f05b806c590 2024-05-03T18:45:28.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:28.092+0000 7f05be12e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05b806bbd0 0x7f05b819f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:28.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:28.092+0000 7f05be12e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05b806bbd0 0x7f05b819f6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42874/0 (socket says 172.21.15.32:42874) 2024-05-03T18:45:28.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:28.092+0000 7f05be12e700 1 -- 172.21.15.32:0/1294506865 learned_addr learned my addr 172.21.15.32:0/1294506865 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:45:28.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:28.092+0000 7f05bd92d700 1 --2- 172.21.15.32:0/1294506865 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f05b806c590 0x7f05b819fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:28.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:28.092+0000 7f05be92f700 1 --2- 172.21.15.32:0/1294506865 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05b8109510 0x7f05b8199900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:28.095 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:28.092+0000 7f05be12e700 1 -- 172.21.15.32:0/1294506865 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f05b806c590 msgr2=0x7f05b819fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:28.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:28.092+0000 7f05be12e700 1 --2- 172.21.15.32:0/1294506865 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f05b806c590 0x7f05b819fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:28.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:28.092+0000 7f05be12e700 1 -- 172.21.15.32:0/1294506865 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05b8109510 msgr2=0x7f05b8199900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:28.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:28.092+0000 7f05be12e700 1 --2- 172.21.15.32:0/1294506865 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05b8109510 0x7f05b8199900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:28.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:28.092+0000 7f05be12e700 1 -- 172.21.15.32:0/1294506865 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05b819a060 con 0x7f05b806bbd0 2024-05-03T18:45:28.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:28.092+0000 7f05be12e700 1 --2- 172.21.15.32:0/1294506865 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05b806bbd0 0x7f05b819f6f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f05b400c9b0 tx=0x7f05b400ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:45:28.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:28.093+0000 7f05af7fe700 1 -- 172.21.15.32:0/1294506865 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05b4007b00 con 0x7f05b806bbd0 2024-05-03T18:45:28.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:28.093+0000 7f05af7fe700 1 -- 172.21.15.32:0/1294506865 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f05b400ce50 con 0x7f05b806bbd0 2024-05-03T18:45:28.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:28.093+0000 7f05c0392700 1 -- 172.21.15.32:0/1294506865 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05b819a350 con 0x7f05b806bbd0 2024-05-03T18:45:28.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:28.093+0000 7f05af7fe700 1 -- 172.21.15.32:0/1294506865 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05b4018660 con 0x7f05b806bbd0 2024-05-03T18:45:28.096 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:28.093+0000 7f05c0392700 1 -- 172.21.15.32:0/1294506865 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05b81a65c0 con 0x7f05b806bbd0 2024-05-03T18:45:28.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:28.094+0000 7f05c0392700 1 -- 172.21.15.32:0/1294506865 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05b806a190 con 0x7f05b806bbd0 2024-05-03T18:45:28.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:28.095+0000 7f05af7fe700 1 -- 172.21.15.32:0/1294506865 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f05b400f450 con 0x7f05b806bbd0 2024-05-03T18:45:28.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:28.095+0000 7f05af7fe700 1 --2- 172.21.15.32:0/1294506865 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f05a40b2910 0x7f05a40b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:28.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:28.096+0000 7f05bd92d700 1 --2- 172.21.15.32:0/1294506865 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f05a40b2910 0x7f05a40b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:28.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:28.096+0000 7f05af7fe700 1 -- 172.21.15.32:0/1294506865 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7979+0+0 (secure 0 0 0) 0x7f05b40e2330 con 0x7f05b806bbd0 2024-05-03T18:45:28.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:28.096+0000 7f05bd92d700 1 --2- 172.21.15.32:0/1294506865 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f05a40b2910 0x7f05a40b4dd0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f05b8079950 tx=0x7f05a800b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:45:28.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:28.099+0000 7f05af7fe700 1 -- 172.21.15.32:0/1294506865 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f05b4014030 con 0x7f05b806bbd0 2024-05-03T18:45:28.320 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:28.319+0000 7f05c0392700 1 -- 172.21.15.32:0/1294506865 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f05b819ae10 con 0x7f05b806bbd0 2024-05-03T18:45:28.321 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:28.320+0000 7f05af7fe700 1 -- 172.21.15.32:0/1294506865 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) ==== 74+0+22148 (secure 0 0 0) 0x7f05b819ae10 con 0x7f05b806bbd0 2024-05-03T18:45:28.322 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-03T18:45:28.322 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":78,"fsid":"f500455e-097b-11ef-bc97-c7b262605968","created":"2024-05-03T18:36:43.750772+0000","modified":"2024-05-03T18:44:51.155086+0000","last_up_change":"2024-05-03T18:43:50.933818+0000","last_in_change":"2024-05-03T18:43:31.804386+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-03T18:40:07.981853+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-03T18:44:44.607622+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-03T18:44:48.077391+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0430c3c0-2c50-471c-9c74-2869a0627c74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":262324956},{"type":"v1","addr":"172.21.15.32:6803","nonce":262324956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":262324956},{"type":"v1","addr":"172.21.15.32:6805","nonce":262324956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":262324956},{"type":"v1","addr":"172.21.15.32:6809","nonce":262324956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":262324956},{"type":"v1","addr":"172.21.15.32:6807","nonce":262324956}]},"public_addr":"172.21.15.32:6803/262324956","cluster_addr":"172.21.15.32:6805/262324956","heartbeat_back_addr":"172.21.15.32:6809/262324956","heartbeat_front_addr":"172.21.15.32:6807/262324956","state":["exists","up"]},{"osd":1,"uuid":"ba8f0b67-1d7d-4d4c-8276-eff664d21f02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1209907302},{"type":"v1","addr":"172.21.15.45:6801","nonce":1209907302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1209907302},{"type":"v1","addr":"172.21.15.45:6803","nonce":1209907302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1209907302},{"type":"v1","addr":"172.21.15.45:6807","nonce":1209907302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1209907302},{"type":"v1","addr":"172.21.15.45:6805","nonce":1209907302}]},"public_addr":"172.21.15.45:6801/1209907302","cluster_addr":"172.21.15.45:6803/1209907302","heartbeat_back_addr":"172.21.15.45:6807/1209907302","heartbeat_front_addr":"172.21.15.45:6805/1209907302","state":["exists","up"]},{"osd":2,"uuid":"5624eabc-be47-4bb8-9d8e-b84734769c83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":2308128926},{"type":"v1","addr":"172.21.15.113:6801","nonce":2308128926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":2308128926},{"type":"v1","addr":"172.21.15.113:6803","nonce":2308128926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":2308128926},{"type":"v1","addr":"172.21.15.113:6807","nonce":2308128926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":2308128926},{"type":"v1","addr":"172.21.15.113:6805","nonce":2308128926}]},"public_addr":"172.21.15.113:6801/2308128926","cluster_addr":"172.21.15.113:6803/2308128926","heartbeat_back_addr":"172.21.15.113:6807/2308128926","heartbeat_front_addr":"172.21.15.113:6805/2308128926","state":["exists","up"]},{"osd":3,"uuid":"62c9c977-afe1-4112-beba-b25af7caa2f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":2606514063},{"type":"v1","addr":"172.21.15.32:6811","nonce":2606514063}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":2606514063},{"type":"v1","addr":"172.21.15.32:6813","nonce":2606514063}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":2606514063},{"type":"v1","addr":"172.21.15.32:6817","nonce":2606514063}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":2606514063},{"type":"v1","addr":"172.21.15.32:6815","nonce":2606514063}]},"public_addr":"172.21.15.32:6811/2606514063","cluster_addr":"172.21.15.32:6813/2606514063","heartbeat_back_addr":"172.21.15.32:6817/2606514063","heartbeat_front_addr":"172.21.15.32:6815/2606514063","state":["exists","up"]},{"osd":4,"uuid":"99ad449f-b0bc-4aa1-b0cf-4a7a458126e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":71932118},{"type":"v1","addr":"172.21.15.45:6809","nonce":71932118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":71932118},{"type":"v1","addr":"172.21.15.45:6811","nonce":71932118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":71932118},{"type":"v1","addr":"172.21.15.45:6815","nonce":71932118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":71932118},{"type":"v1","addr":"172.21.15.45:6813","nonce":71932118}]},"public_addr":"172.21.15.45:6809/71932118","cluster_addr":"172.21.15.45:6811/71932118","heartbeat_back_addr":"172.21.15.45:6815/71932118","heartbeat_front_addr":"172.21.15.45:6813/71932118","state":["exists","up"]},{"osd":5,"uuid":"a6688ade-47d7-4a52-807f-3c880a8ffcbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":4085018597},{"type":"v1","addr":"172.21.15.113:6809","nonce":4085018597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":4085018597},{"type":"v1","addr":"172.21.15.113:6811","nonce":4085018597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":4085018597},{"type":"v1","addr":"172.21.15.113:6815","nonce":4085018597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":4085018597},{"type":"v1","addr":"172.21.15.113:6813","nonce":4085018597}]},"public_addr":"172.21.15.113:6809/4085018597","cluster_addr":"172.21.15.113:6811/4085018597","heartbeat_back_addr":"172.21.15.113:6815/4085018597","heartbeat_front_addr":"172.21.15.113:6813/4085018597","state":["exists","up"]},{"osd":6,"uuid":"1d9ebda9-55c7-473a-9dc0-fc60845bbfe9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":791407796},{"type":"v1","addr":"172.21.15.32:6819","nonce":791407796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":791407796},{"type":"v1","addr":"172.21.15.32:6821","nonce":791407796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":791407796},{"type":"v1","addr":"172.21.15.32:6825","nonce":791407796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":791407796},{"type":"v1","addr":"172.21.15.32:6823","nonce":791407796}]},"public_addr":"172.21.15.32:6819/791407796","cluster_addr":"172.21.15.32:6821/791407796","heartbeat_back_addr":"172.21.15.32:6825/791407796","heartbeat_front_addr":"172.21.15.32:6823/791407796","state":["exists","up"]},{"osd":7,"uuid":"988cce65-23ed-4814-ac09-177eb2f89fe1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1032572438},{"type":"v1","addr":"172.21.15.45:6817","nonce":1032572438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1032572438},{"type":"v1","addr":"172.21.15.45:6819","nonce":1032572438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1032572438},{"type":"v1","addr":"172.21.15.45:6823","nonce":1032572438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1032572438},{"type":"v1","addr":"172.21.15.45:6821","nonce":1032572438}]},"public_addr":"172.21.15.45:6817/1032572438","cluster_addr":"172.21.15.45:6819/1032572438","heartbeat_back_addr":"172.21.15.45:6823/1032572438","heartbeat_front_addr":"172.21.15.45:6821/1032572438","state":["exists","up"]},{"osd":8,"uuid":"816e8ce8-5afa-4b52-89be-f7793ed12a0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3569181558},{"type":"v1","addr":"172.21.15.113:6817","nonce":3569181558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3569181558},{"type":"v1","addr":"172.21.15.113:6819","nonce":3569181558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3569181558},{"type":"v1","addr":"172.21.15.113:6823","nonce":3569181558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3569181558},{"type":"v1","addr":"172.21.15.113:6821","nonce":3569181558}]},"public_addr":"172.21.15.113:6817/3569181558","cluster_addr":"172.21.15.113:6819/3569181558","heartbeat_back_addr":"172.21.15.113:6823/3569181558","heartbeat_front_addr":"172.21.15.113:6821/3569181558","state":["exists","up"]},{"osd":9,"uuid":"1d35c435-36b8-40e8-a2d4-0efe904f1107","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":4290196345},{"type":"v1","addr":"172.21.15.32:6827","nonce":4290196345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":4290196345},{"type":"v1","addr":"172.21.15.32:6829","nonce":4290196345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":4290196345},{"type":"v1","addr":"172.21.15.32:6833","nonce":4290196345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":4290196345},{"type":"v1","addr":"172.21.15.32:6831","nonce":4290196345}]},"public_addr":"172.21.15.32:6827/4290196345","cluster_addr":"172.21.15.32:6829/4290196345","heartbeat_back_addr":"172.21.15.32:6833/4290196345","heartbeat_front_addr":"172.21.15.32:6831/4290196345","state":["exists","up"]},{"osd":10,"uuid":"9af7879e-aa8d-4ef8-b289-4b145b614820","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1817064361},{"type":"v1","addr":"172.21.15.45:6825","nonce":1817064361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1817064361},{"type":"v1","addr":"172.21.15.45:6827","nonce":1817064361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1817064361},{"type":"v1","addr":"172.21.15.45:6831","nonce":1817064361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1817064361},{"type":"v1","addr":"172.21.15.45:6829","nonce":1817064361}]},"public_addr":"172.21.15.45:6825/1817064361","cluster_addr":"172.21.15.45:6827/1817064361","heartbeat_back_addr":"172.21.15.45:6831/1817064361","heartbeat_front_addr":"172.21.15.45:6829/1817064361","state":["exists","up"]},{"osd":11,"uuid":"22981709-1f9d-46f6-892a-5678394a2e1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":703464306},{"type":"v1","addr":"172.21.15.113:6825","nonce":703464306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":703464306},{"type":"v1","addr":"172.21.15.113:6827","nonce":703464306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":703464306},{"type":"v1","addr":"172.21.15.113:6831","nonce":703464306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":703464306},{"type":"v1","addr":"172.21.15.113:6829","nonce":703464306}]},"public_addr":"172.21.15.113:6825/703464306","cluster_addr":"172.21.15.113:6827/703464306","heartbeat_back_addr":"172.21.15.113:6831/703464306","heartbeat_front_addr":"172.21.15.113:6829/703464306","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:39:18.455575+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:39:42.234466+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:40:05.843072+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:40:30.822533+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:40:54.952487+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:41:18.750184+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:41:43.403326+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:42:08.285730+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:42:32.861740+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:42:59.307277+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:43:24.509691+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:43:49.850077+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:0/743758274":"2024-05-04T18:37:45.879602+0000","172.21.15.32:6801/2019702098":"2024-05-04T18:37:45.879602+0000","172.21.15.32:6800/2019702098":"2024-05-04T18:37:45.879602+0000","172.21.15.32:0/3192385439":"2024-05-04T18:37:18.492319+0000","172.21.15.32:0/1450308494":"2024-05-04T18:37:18.492319+0000","172.21.15.32:0/3957699240":"2024-05-04T18:37:45.879602+0000","172.21.15.32:0/3310503393":"2024-05-04T18:37:18.492319+0000","172.21.15.32:6801/3945405684":"2024-05-04T18:37:18.492319+0000","172.21.15.32:6800/3945405684":"2024-05-04T18:37:18.492319+0000","172.21.15.32:0/2471164468":"2024-05-04T18:37:45.879602+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-03T18:45:28.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:28.323+0000 7f05c0392700 1 -- 172.21.15.32:0/1294506865 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f05a40b2910 msgr2=0x7f05a40b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:28.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:28.323+0000 7f05c0392700 1 --2- 172.21.15.32:0/1294506865 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f05a40b2910 0x7f05a40b4dd0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f05b8079950 tx=0x7f05a800b3c0 comp rx=0 tx=0).stop 2024-05-03T18:45:28.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:28.323+0000 7f05c0392700 1 -- 172.21.15.32:0/1294506865 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05b806bbd0 msgr2=0x7f05b819f6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:28.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:28.323+0000 7f05c0392700 1 --2- 172.21.15.32:0/1294506865 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05b806bbd0 0x7f05b819f6f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f05b400c9b0 tx=0x7f05b400ccc0 comp rx=0 tx=0).stop 2024-05-03T18:45:28.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:28.323+0000 7f05c0392700 1 -- 172.21.15.32:0/1294506865 shutdown_connections 2024-05-03T18:45:28.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:28.323+0000 7f05c0392700 1 --2- 172.21.15.32:0/1294506865 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f05a40b2910 0x7f05a40b4dd0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:28.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:28.323+0000 7f05c0392700 1 --2- 172.21.15.32:0/1294506865 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05b8109510 0x7f05b8199900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:28.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:28.323+0000 7f05c0392700 1 --2- 172.21.15.32:0/1294506865 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05b806bbd0 0x7f05b819f6f0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:28.324 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:28.323+0000 7f05c0392700 1 --2- 172.21.15.32:0/1294506865 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f05b806c590 0x7f05b819fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:28.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:28.323+0000 7f05c0392700 1 -- 172.21.15.32:0/1294506865 >> 172.21.15.32:0/1294506865 conn(0x7f05b8079350 msgr2=0x7f05b8101090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:45:28.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:28.323+0000 7f05c0392700 1 -- 172.21.15.32:0/1294506865 shutdown_connections 2024-05-03T18:45:28.325 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:28.323+0000 7f05c0392700 1 -- 172.21.15.32:0/1294506865 wait complete. 2024-05-03T18:45:28.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:45:28 smithi113 ceph-mon[33540]: from='client.? 172.21.15.32:0/1294506865' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T18:45:28.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:45:28 smithi032 ceph-mon[29763]: from='client.? 172.21.15.32:0/1294506865' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T18:45:28.947 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-05-03T18:45:28.947 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid f500455e-097b-11ef-bc97-c7b262605968 -- ceph pg dump --format=json 2024-05-03T18:45:28.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:45:28 smithi045 ceph-mon[33299]: from='client.? 172.21.15.32:0/1294506865' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T18:45:29.195 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:45:29.567 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:45:29 smithi032 ceph-mon[29763]: pgmap v299: 97 pgs: 97 active+clean; 630 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 4.3 KiB/s wr, 9 op/s 2024-05-03T18:45:29.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:45:29 smithi113 ceph-mon[33540]: pgmap v299: 97 pgs: 97 active+clean; 630 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 4.3 KiB/s wr, 9 op/s 2024-05-03T18:45:29.997 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:45:29 smithi045 ceph-mon[33299]: pgmap v299: 97 pgs: 97 active+clean; 630 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 4.3 KiB/s wr, 9 op/s 2024-05-03T18:45:30.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:30.790+0000 7fd147707700 1 -- 172.21.15.32:0/816354937 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd14006c110 msgr2=0x7fd14006c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:30.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:30.790+0000 7fd147707700 1 --2- 172.21.15.32:0/816354937 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd14006c110 0x7fd14006c590 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fd130009990 tx=0x7fd130009ca0 comp rx=0 tx=0).stop 2024-05-03T18:45:30.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:30.790+0000 7fd147707700 1 -- 172.21.15.32:0/816354937 shutdown_connections 2024-05-03T18:45:30.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:30.790+0000 7fd147707700 1 --2- 172.21.15.32:0/816354937 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd140105ed0 0x7fd140106380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:30.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:30.790+0000 7fd147707700 1 --2- 172.21.15.32:0/816354937 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd1401083c0 0x7fd14006bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:30.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:30.790+0000 7fd147707700 1 --2- 172.21.15.32:0/816354937 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd14006c110 0x7fd14006c590 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:30.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:30.790+0000 7fd147707700 1 -- 172.21.15.32:0/816354937 >> 172.21.15.32:0/816354937 conn(0x7fd140079350 msgr2=0x7fd140079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:45:30.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:30.790+0000 7fd147707700 1 -- 172.21.15.32:0/816354937 shutdown_connections 2024-05-03T18:45:30.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:30.790+0000 7fd147707700 1 -- 172.21.15.32:0/816354937 wait complete. 2024-05-03T18:45:30.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:30.791+0000 7fd147707700 1 Processor -- start 2024-05-03T18:45:30.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:30.791+0000 7fd147707700 1 -- start start 2024-05-03T18:45:30.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:30.791+0000 7fd147707700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd14006c110 0x7fd1401a0130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:30.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:30.791+0000 7fd147707700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd140105ed0 0x7fd1401a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:30.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:30.791+0000 7fd147707700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd1401083c0 0x7fd1401a4a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:30.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:30.791+0000 7fd147707700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd140119c20 con 0x7fd140105ed0 2024-05-03T18:45:30.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:30.791+0000 7fd147707700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd140119aa0 con 0x7fd14006c110 2024-05-03T18:45:30.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:30.791+0000 7fd147707700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fd140119da0 con 0x7fd1401083c0 2024-05-03T18:45:30.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:30.791+0000 7fd144ca2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd140105ed0 0x7fd1401a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:30.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:30.791+0000 7fd144ca2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd140105ed0 0x7fd1401a0670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42888/0 (socket says 172.21.15.32:42888) 2024-05-03T18:45:30.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:30.791+0000 7fd144ca2700 1 -- 172.21.15.32:0/900578285 learned_addr learned my addr 172.21.15.32:0/900578285 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:45:30.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:30.791+0000 7fd1454a3700 1 --2- 172.21.15.32:0/900578285 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd14006c110 0x7fd1401a0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:30.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:30.791+0000 7fd145ca4700 1 --2- 172.21.15.32:0/900578285 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd1401083c0 0x7fd1401a4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:30.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:30.791+0000 7fd144ca2700 1 -- 172.21.15.32:0/900578285 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd1401083c0 msgr2=0x7fd1401a4a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:30.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:30.791+0000 7fd144ca2700 1 --2- 172.21.15.32:0/900578285 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd1401083c0 0x7fd1401a4a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:30.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:30.791+0000 7fd144ca2700 1 -- 172.21.15.32:0/900578285 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd14006c110 msgr2=0x7fd1401a0130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:30.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:30.791+0000 7fd144ca2700 1 --2- 172.21.15.32:0/900578285 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd14006c110 0x7fd1401a0130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:30.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:30.791+0000 7fd144ca2700 1 -- 172.21.15.32:0/900578285 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1401a5140 con 0x7fd140105ed0 2024-05-03T18:45:30.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:30.791+0000 7fd144ca2700 1 --2- 172.21.15.32:0/900578285 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd140105ed0 0x7fd1401a0670 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd1300057c0 tx=0x7fd1300057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:45:30.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:30.792+0000 7fd1367fc700 1 -- 172.21.15.32:0/900578285 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd130027070 con 0x7fd140105ed0 2024-05-03T18:45:30.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:30.792+0000 7fd147707700 1 -- 172.21.15.32:0/900578285 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1401a53d0 con 0x7fd140105ed0 2024-05-03T18:45:30.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:30.792+0000 7fd1367fc700 1 -- 172.21.15.32:0/900578285 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd130003f20 con 0x7fd140105ed0 2024-05-03T18:45:30.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:30.792+0000 7fd1367fc700 1 -- 172.21.15.32:0/900578285 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd130022b30 con 0x7fd140105ed0 2024-05-03T18:45:30.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:30.792+0000 7fd147707700 1 -- 172.21.15.32:0/900578285 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1401accb0 con 0x7fd140105ed0 2024-05-03T18:45:30.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:30.793+0000 7fd147707700 1 -- 172.21.15.32:0/900578285 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd14006a190 con 0x7fd140105ed0 2024-05-03T18:45:30.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:30.794+0000 7fd1367fc700 1 -- 172.21.15.32:0/900578285 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fd13002a070 con 0x7fd140105ed0 2024-05-03T18:45:30.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:30.795+0000 7fd1367fc700 1 --2- 172.21.15.32:0/900578285 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fd12c0b2910 0x7fd12c0b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:30.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:30.795+0000 7fd1454a3700 1 --2- 172.21.15.32:0/900578285 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fd12c0b2910 0x7fd12c0b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:30.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:30.795+0000 7fd1367fc700 1 -- 172.21.15.32:0/900578285 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7979+0+0 (secure 0 0 0) 0x7fd1300e3680 con 0x7fd140105ed0 2024-05-03T18:45:30.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:30.795+0000 7fd1454a3700 1 --2- 172.21.15.32:0/900578285 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fd12c0b2910 0x7fd12c0b4dd0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fd13c005950 tx=0x7fd13c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:45:30.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:30.798+0000 7fd1367fc700 1 -- 172.21.15.32:0/900578285 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fd130031020 con 0x7fd140105ed0 2024-05-03T18:45:31.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:31.012+0000 7fd147707700 1 -- 172.21.15.32:0/900578285 --> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd1401055c0 con 0x7fd12c0b2910 2024-05-03T18:45:31.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:31.020+0000 7fd1367fc700 1 -- 172.21.15.32:0/900578285 <== mgr.14152 v2:172.21.15.32:6800/1796547904 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258695 (secure 0 0 0) 0x7fd1401055c0 con 0x7fd12c0b2910 2024-05-03T18:45:31.025 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-03T18:45:31.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:31.023+0000 7fd147707700 1 -- 172.21.15.32:0/900578285 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fd12c0b2910 msgr2=0x7fd12c0b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:31.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:31.023+0000 7fd147707700 1 --2- 172.21.15.32:0/900578285 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fd12c0b2910 0x7fd12c0b4dd0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fd13c005950 tx=0x7fd13c00b410 comp rx=0 tx=0).stop 2024-05-03T18:45:31.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:31.023+0000 7fd147707700 1 -- 172.21.15.32:0/900578285 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd140105ed0 msgr2=0x7fd1401a0670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:31.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:31.023+0000 7fd147707700 1 --2- 172.21.15.32:0/900578285 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd140105ed0 0x7fd1401a0670 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd1300057c0 tx=0x7fd1300057f0 comp rx=0 tx=0).stop 2024-05-03T18:45:31.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:31.023+0000 7fd147707700 1 -- 172.21.15.32:0/900578285 shutdown_connections 2024-05-03T18:45:31.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:31.023+0000 7fd147707700 1 --2- 172.21.15.32:0/900578285 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fd12c0b2910 0x7fd12c0b4dd0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:31.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:31.023+0000 7fd147707700 1 --2- 172.21.15.32:0/900578285 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd1401083c0 0x7fd1401a4a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:31.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:31.023+0000 7fd147707700 1 --2- 172.21.15.32:0/900578285 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd14006c110 0x7fd1401a0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:31.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:31.023+0000 7fd147707700 1 --2- 172.21.15.32:0/900578285 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd140105ed0 0x7fd1401a0670 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:31.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:31.023+0000 7fd147707700 1 -- 172.21.15.32:0/900578285 >> 172.21.15.32:0/900578285 conn(0x7fd140079350 msgr2=0x7fd140104eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:45:31.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:31.023+0000 7fd147707700 1 -- 172.21.15.32:0/900578285 shutdown_connections 2024-05-03T18:45:31.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:31.023+0000 7fd147707700 1 -- 172.21.15.32:0/900578285 wait complete. 2024-05-03T18:45:31.028 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-05-03T18:45:31.775 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:45:31 smithi032 ceph-mon[29763]: pgmap v300: 97 pgs: 97 active+clean; 630 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 2.9 KiB/s wr, 6 op/s 2024-05-03T18:45:31.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:45:31 smithi113 ceph-mon[33540]: pgmap v300: 97 pgs: 97 active+clean; 630 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 2.9 KiB/s wr, 6 op/s 2024-05-03T18:45:31.806 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":300,"stamp":"2024-05-03T18:45:29.973348+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":645298,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":232,"num_read_kb":199,"num_write":282,"num_write_kb":2250,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":228,"ondisk_log_size":228,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":369412,"kb_used_data":11940,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124572412,"statfs":{"total":1151940427776,"available":1151562149888,"internally_reserved":0,"allocated":12226560,"data_stored":6827522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":365933883},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":28482,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":63,"num_read_kb":59,"num_write":20,"num_write_kb":35,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003317"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:44:51.430010+0000","last_change":"2024-05-03T18:44:49.430467+0000","last_active":"2024-05-03T18:44:51.430010+0000","last_peered":"2024-05-03T18:44:51.430010+0000","last_clean":"2024-05-03T18:44:51.430010+0000","last_became_active":"2024-05-03T18:44:49.430147+0000","last_became_peered":"2024-05-03T18:44:49.430147+0000","last_unstale":"2024-05-03T18:44:51.430010+0000","last_undegraded":"2024-05-03T18:44:51.430010+0000","last_fullsized":"2024-05-03T18:44:51.430010+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:49:22.725690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:44:51.432156+0000","last_change":"2024-05-03T18:44:49.432057+0000","last_active":"2024-05-03T18:44:51.432156+0000","last_peered":"2024-05-03T18:44:51.432156+0000","last_clean":"2024-05-03T18:44:51.432156+0000","last_became_active":"2024-05-03T18:44:49.430867+0000","last_became_peered":"2024-05-03T18:44:49.430867+0000","last_unstale":"2024-05-03T18:44:51.432156+0000","last_undegraded":"2024-05-03T18:44:51.432156+0000","last_fullsized":"2024-05-03T18:44:51.432156+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:31:16.470041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:44:51.432046+0000","last_change":"2024-05-03T18:44:49.433622+0000","last_active":"2024-05-03T18:44:51.432046+0000","last_peered":"2024-05-03T18:44:51.432046+0000","last_clean":"2024-05-03T18:44:51.432046+0000","last_became_active":"2024-05-03T18:44:49.431803+0000","last_became_peered":"2024-05-03T18:44:49.431803+0000","last_unstale":"2024-05-03T18:44:51.432046+0000","last_undegraded":"2024-05-03T18:44:51.432046+0000","last_fullsized":"2024-05-03T18:44:51.432046+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:40:18.492498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:44:51.423005+0000","last_change":"2024-05-03T18:44:49.437469+0000","last_active":"2024-05-03T18:44:51.423005+0000","last_peered":"2024-05-03T18:44:51.423005+0000","last_clean":"2024-05-03T18:44:51.423005+0000","last_became_active":"2024-05-03T18:44:49.435519+0000","last_became_peered":"2024-05-03T18:44:49.435519+0000","last_unstale":"2024-05-03T18:44:51.423005+0000","last_undegraded":"2024-05-03T18:44:51.423005+0000","last_fullsized":"2024-05-03T18:44:51.423005+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:11:07.246146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:44:51.434741+0000","last_change":"2024-05-03T18:44:49.435115+0000","last_active":"2024-05-03T18:44:51.434741+0000","last_peered":"2024-05-03T18:44:51.434741+0000","last_clean":"2024-05-03T18:44:51.434741+0000","last_became_active":"2024-05-03T18:44:49.434820+0000","last_became_peered":"2024-05-03T18:44:49.434820+0000","last_unstale":"2024-05-03T18:44:51.434741+0000","last_undegraded":"2024-05-03T18:44:51.434741+0000","last_fullsized":"2024-05-03T18:44:51.434741+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:32:24.817201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:44:51.430766+0000","last_change":"2024-05-03T18:44:49.432143+0000","last_active":"2024-05-03T18:44:51.430766+0000","last_peered":"2024-05-03T18:44:51.430766+0000","last_clean":"2024-05-03T18:44:51.430766+0000","last_became_active":"2024-05-03T18:44:49.431742+0000","last_became_peered":"2024-05-03T18:44:49.431742+0000","last_unstale":"2024-05-03T18:44:51.430766+0000","last_undegraded":"2024-05-03T18:44:51.430766+0000","last_fullsized":"2024-05-03T18:44:51.430766+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:25:36.384277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:44:51.429846+0000","last_change":"2024-05-03T18:44:49.428955+0000","last_active":"2024-05-03T18:44:51.429846+0000","last_peered":"2024-05-03T18:44:51.429846+0000","last_clean":"2024-05-03T18:44:51.429846+0000","last_became_active":"2024-05-03T18:44:49.428587+0000","last_became_peered":"2024-05-03T18:44:49.428587+0000","last_unstale":"2024-05-03T18:44:51.429846+0000","last_undegraded":"2024-05-03T18:44:51.429846+0000","last_fullsized":"2024-05-03T18:44:51.429846+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:14:34.637035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:44:51.430027+0000","last_change":"2024-05-03T18:44:49.432804+0000","last_active":"2024-05-03T18:44:51.430027+0000","last_peered":"2024-05-03T18:44:51.430027+0000","last_clean":"2024-05-03T18:44:51.430027+0000","last_became_active":"2024-05-03T18:44:49.431828+0000","last_became_peered":"2024-05-03T18:44:49.431828+0000","last_unstale":"2024-05-03T18:44:51.430027+0000","last_undegraded":"2024-05-03T18:44:51.430027+0000","last_fullsized":"2024-05-03T18:44:51.430027+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:06:40.810972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:44:51.435402+0000","last_change":"2024-05-03T18:44:49.428919+0000","last_active":"2024-05-03T18:44:51.435402+0000","last_peered":"2024-05-03T18:44:51.435402+0000","last_clean":"2024-05-03T18:44:51.435402+0000","last_became_active":"2024-05-03T18:44:49.428350+0000","last_became_peered":"2024-05-03T18:44:49.428350+0000","last_unstale":"2024-05-03T18:44:51.435402+0000","last_undegraded":"2024-05-03T18:44:51.435402+0000","last_fullsized":"2024-05-03T18:44:51.435402+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T18:50:09.895485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:44:51.431609+0000","last_change":"2024-05-03T18:44:49.431765+0000","last_active":"2024-05-03T18:44:51.431609+0000","last_peered":"2024-05-03T18:44:51.431609+0000","last_clean":"2024-05-03T18:44:51.431609+0000","last_became_active":"2024-05-03T18:44:49.431417+0000","last_became_peered":"2024-05-03T18:44:49.431417+0000","last_unstale":"2024-05-03T18:44:51.431609+0000","last_undegraded":"2024-05-03T18:44:51.431609+0000","last_fullsized":"2024-05-03T18:44:51.431609+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:24:58.097954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:44:51.430506+0000","last_change":"2024-05-03T18:44:49.434154+0000","last_active":"2024-05-03T18:44:51.430506+0000","last_peered":"2024-05-03T18:44:51.430506+0000","last_clean":"2024-05-03T18:44:51.430506+0000","last_became_active":"2024-05-03T18:44:49.433806+0000","last_became_peered":"2024-05-03T18:44:49.433806+0000","last_unstale":"2024-05-03T18:44:51.430506+0000","last_undegraded":"2024-05-03T18:44:51.430506+0000","last_fullsized":"2024-05-03T18:44:51.430506+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:52:39.233556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:44:51.422984+0000","last_change":"2024-05-03T18:44:49.435385+0000","last_active":"2024-05-03T18:44:51.422984+0000","last_peered":"2024-05-03T18:44:51.422984+0000","last_clean":"2024-05-03T18:44:51.422984+0000","last_became_active":"2024-05-03T18:44:49.434727+0000","last_became_peered":"2024-05-03T18:44:49.434727+0000","last_unstale":"2024-05-03T18:44:51.422984+0000","last_undegraded":"2024-05-03T18:44:51.422984+0000","last_fullsized":"2024-05-03T18:44:51.422984+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:08:45.986371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:44:51.422322+0000","last_change":"2024-05-03T18:44:49.428536+0000","last_active":"2024-05-03T18:44:51.422322+0000","last_peered":"2024-05-03T18:44:51.422322+0000","last_clean":"2024-05-03T18:44:51.422322+0000","last_became_active":"2024-05-03T18:44:49.427630+0000","last_became_peered":"2024-05-03T18:44:49.427630+0000","last_unstale":"2024-05-03T18:44:51.422322+0000","last_undegraded":"2024-05-03T18:44:51.422322+0000","last_fullsized":"2024-05-03T18:44:51.422322+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:24:09.622403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:44:51.429974+0000","last_change":"2024-05-03T18:44:49.429236+0000","last_active":"2024-05-03T18:44:51.429974+0000","last_peered":"2024-05-03T18:44:51.429974+0000","last_clean":"2024-05-03T18:44:51.429974+0000","last_became_active":"2024-05-03T18:44:49.428942+0000","last_became_peered":"2024-05-03T18:44:49.428942+0000","last_unstale":"2024-05-03T18:44:51.429974+0000","last_undegraded":"2024-05-03T18:44:51.429974+0000","last_fullsized":"2024-05-03T18:44:51.429974+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:16:44.241736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:44:51.429575+0000","last_change":"2024-05-03T18:44:49.433388+0000","last_active":"2024-05-03T18:44:51.429575+0000","last_peered":"2024-05-03T18:44:51.429575+0000","last_clean":"2024-05-03T18:44:51.429575+0000","last_became_active":"2024-05-03T18:44:49.431869+0000","last_became_peered":"2024-05-03T18:44:49.431869+0000","last_unstale":"2024-05-03T18:44:51.429575+0000","last_undegraded":"2024-05-03T18:44:51.429575+0000","last_fullsized":"2024-05-03T18:44:51.429575+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:36:52.803391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:44:51.433507+0000","last_change":"2024-05-03T18:44:49.425781+0000","last_active":"2024-05-03T18:44:51.433507+0000","last_peered":"2024-05-03T18:44:51.433507+0000","last_clean":"2024-05-03T18:44:51.433507+0000","last_became_active":"2024-05-03T18:44:49.425248+0000","last_became_peered":"2024-05-03T18:44:49.425248+0000","last_unstale":"2024-05-03T18:44:51.433507+0000","last_undegraded":"2024-05-03T18:44:51.433507+0000","last_fullsized":"2024-05-03T18:44:51.433507+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:27:25.017459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:44:51.430174+0000","last_change":"2024-05-03T18:44:49.430099+0000","last_active":"2024-05-03T18:44:51.430174+0000","last_peered":"2024-05-03T18:44:51.430174+0000","last_clean":"2024-05-03T18:44:51.430174+0000","last_became_active":"2024-05-03T18:44:49.429685+0000","last_became_peered":"2024-05-03T18:44:49.429685+0000","last_unstale":"2024-05-03T18:44:51.430174+0000","last_undegraded":"2024-05-03T18:44:51.430174+0000","last_fullsized":"2024-05-03T18:44:51.430174+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:13:02.658508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:44:51.431464+0000","last_change":"2024-05-03T18:44:49.427176+0000","last_active":"2024-05-03T18:44:51.431464+0000","last_peered":"2024-05-03T18:44:51.431464+0000","last_clean":"2024-05-03T18:44:51.431464+0000","last_became_active":"2024-05-03T18:44:49.426546+0000","last_became_peered":"2024-05-03T18:44:49.426546+0000","last_unstale":"2024-05-03T18:44:51.431464+0000","last_undegraded":"2024-05-03T18:44:51.431464+0000","last_fullsized":"2024-05-03T18:44:51.431464+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:29:22.343376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:44:51.432297+0000","last_change":"2024-05-03T18:44:49.435037+0000","last_active":"2024-05-03T18:44:51.432297+0000","last_peered":"2024-05-03T18:44:51.432297+0000","last_clean":"2024-05-03T18:44:51.432297+0000","last_became_active":"2024-05-03T18:44:49.434693+0000","last_became_peered":"2024-05-03T18:44:49.434693+0000","last_unstale":"2024-05-03T18:44:51.432297+0000","last_undegraded":"2024-05-03T18:44:51.432297+0000","last_fullsized":"2024-05-03T18:44:51.432297+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:26:33.923608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"78'9","reported_seq":31,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:45:03.483324+0000","last_change":"2024-05-03T18:44:46.402606+0000","last_active":"2024-05-03T18:45:03.483324+0000","last_peered":"2024-05-03T18:45:03.483324+0000","last_clean":"2024-05-03T18:45:03.483324+0000","last_became_active":"2024-05-03T18:44:46.402245+0000","last_became_peered":"2024-05-03T18:44:46.402245+0000","last_unstale":"2024-05-03T18:45:03.483324+0000","last_undegraded":"2024-05-03T18:45:03.483324+0000","last_fullsized":"2024-05-03T18:45:03.483324+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_clean_scrub_stamp":"2024-05-03T18:44:45.380544+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:29:33.726119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:44:51.430853+0000","last_change":"2024-05-03T18:44:49.435279+0000","last_active":"2024-05-03T18:44:51.430853+0000","last_peered":"2024-05-03T18:44:51.430853+0000","last_clean":"2024-05-03T18:44:51.430853+0000","last_became_active":"2024-05-03T18:44:49.434779+0000","last_became_peered":"2024-05-03T18:44:49.434779+0000","last_unstale":"2024-05-03T18:44:51.430853+0000","last_undegraded":"2024-05-03T18:44:51.430853+0000","last_fullsized":"2024-05-03T18:44:51.430853+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:56:56.762051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"78'2","reported_seq":23,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:44:51.434991+0000","last_change":"2024-05-03T18:44:46.644899+0000","last_active":"2024-05-03T18:44:51.434991+0000","last_peered":"2024-05-03T18:44:51.434991+0000","last_clean":"2024-05-03T18:44:51.434991+0000","last_became_active":"2024-05-03T18:44:46.644508+0000","last_became_peered":"2024-05-03T18:44:46.644508+0000","last_unstale":"2024-05-03T18:44:51.434991+0000","last_undegraded":"2024-05-03T18:44:51.434991+0000","last_fullsized":"2024-05-03T18:44:51.434991+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_clean_scrub_stamp":"2024-05-03T18:44:45.380544+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T06:38:09.379634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:44:51.432839+0000","last_change":"2024-05-03T18:44:49.429720+0000","last_active":"2024-05-03T18:44:51.432839+0000","last_peered":"2024-05-03T18:44:51.432839+0000","last_clean":"2024-05-03T18:44:51.432839+0000","last_became_active":"2024-05-03T18:44:49.428399+0000","last_became_peered":"2024-05-03T18:44:49.428399+0000","last_unstale":"2024-05-03T18:44:51.432839+0000","last_undegraded":"2024-05-03T18:44:51.432839+0000","last_fullsized":"2024-05-03T18:44:51.432839+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:17:57.325926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"78'4","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:45:03.483080+0000","last_change":"2024-05-03T18:44:46.410397+0000","last_active":"2024-05-03T18:45:03.483080+0000","last_peered":"2024-05-03T18:45:03.483080+0000","last_clean":"2024-05-03T18:45:03.483080+0000","last_became_active":"2024-05-03T18:44:46.410030+0000","last_became_peered":"2024-05-03T18:44:46.410030+0000","last_unstale":"2024-05-03T18:45:03.483080+0000","last_undegraded":"2024-05-03T18:45:03.483080+0000","last_fullsized":"2024-05-03T18:45:03.483080+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_clean_scrub_stamp":"2024-05-03T18:44:45.380544+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:15:26.130070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:44:51.433958+0000","last_change":"2024-05-03T18:44:49.430335+0000","last_active":"2024-05-03T18:44:51.433958+0000","last_peered":"2024-05-03T18:44:51.433958+0000","last_clean":"2024-05-03T18:44:51.433958+0000","last_became_active":"2024-05-03T18:44:49.429991+0000","last_became_peered":"2024-05-03T18:44:49.429991+0000","last_unstale":"2024-05-03T18:44:51.433958+0000","last_undegraded":"2024-05-03T18:44:51.433958+0000","last_fullsized":"2024-05-03T18:44:51.433958+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:06:28.718138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"78'2","reported_seq":23,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:44:51.438736+0000","last_change":"2024-05-03T18:44:46.406552+0000","last_active":"2024-05-03T18:44:51.438736+0000","last_peered":"2024-05-03T18:44:51.438736+0000","last_clean":"2024-05-03T18:44:51.438736+0000","last_became_active":"2024-05-03T18:44:46.406160+0000","last_became_peered":"2024-05-03T18:44:46.406160+0000","last_unstale":"2024-05-03T18:44:51.438736+0000","last_undegraded":"2024-05-03T18:44:51.438736+0000","last_fullsized":"2024-05-03T18:44:51.438736+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_clean_scrub_stamp":"2024-05-03T18:44:45.380544+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:53:13.769225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:44:51.423057+0000","last_change":"2024-05-03T18:44:49.441301+0000","last_active":"2024-05-03T18:44:51.423057+0000","last_peered":"2024-05-03T18:44:51.423057+0000","last_clean":"2024-05-03T18:44:51.423057+0000","last_became_active":"2024-05-03T18:44:49.440936+0000","last_became_peered":"2024-05-03T18:44:49.440936+0000","last_unstale":"2024-05-03T18:44:51.423057+0000","last_undegraded":"2024-05-03T18:44:51.423057+0000","last_fullsized":"2024-05-03T18:44:51.423057+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:55:41.257325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"78'6","reported_seq":51,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:45:24.521364+0000","last_change":"2024-05-03T18:44:46.404827+0000","last_active":"2024-05-03T18:45:24.521364+0000","last_peered":"2024-05-03T18:45:24.521364+0000","last_clean":"2024-05-03T18:45:24.521364+0000","last_became_active":"2024-05-03T18:44:46.404455+0000","last_became_peered":"2024-05-03T18:44:46.404455+0000","last_unstale":"2024-05-03T18:45:24.521364+0000","last_undegraded":"2024-05-03T18:45:24.521364+0000","last_fullsized":"2024-05-03T18:45:24.521364+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_clean_scrub_stamp":"2024-05-03T18:44:45.380544+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:23:32.855160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":24,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:44:51.434308+0000","last_change":"2024-05-03T18:44:49.433656+0000","last_active":"2024-05-03T18:44:51.434308+0000","last_peered":"2024-05-03T18:44:51.434308+0000","last_clean":"2024-05-03T18:44:51.434308+0000","last_became_active":"2024-05-03T18:44:49.433326+0000","last_became_peered":"2024-05-03T18:44:49.433326+0000","last_unstale":"2024-05-03T18:44:51.434308+0000","last_undegraded":"2024-05-03T18:44:51.434308+0000","last_fullsized":"2024-05-03T18:44:51.434308+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:44:56.937334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"78'3","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:45:03.481450+0000","last_change":"2024-05-03T18:44:46.403960+0000","last_active":"2024-05-03T18:45:03.481450+0000","last_peered":"2024-05-03T18:45:03.481450+0000","last_clean":"2024-05-03T18:45:03.481450+0000","last_became_active":"2024-05-03T18:44:46.403560+0000","last_became_peered":"2024-05-03T18:44:46.403560+0000","last_unstale":"2024-05-03T18:45:03.481450+0000","last_undegraded":"2024-05-03T18:45:03.481450+0000","last_fullsized":"2024-05-03T18:45:03.481450+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_clean_scrub_stamp":"2024-05-03T18:44:45.380544+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:58:32.823640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:44:51.423234+0000","last_change":"2024-05-03T18:44:49.438621+0000","last_active":"2024-05-03T18:44:51.423234+0000","last_peered":"2024-05-03T18:44:51.423234+0000","last_clean":"2024-05-03T18:44:51.423234+0000","last_became_active":"2024-05-03T18:44:49.438288+0000","last_became_peered":"2024-05-03T18:44:49.438288+0000","last_unstale":"2024-05-03T18:44:51.423234+0000","last_undegraded":"2024-05-03T18:44:51.423234+0000","last_fullsized":"2024-05-03T18:44:51.423234+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:15:16.421982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"78'2","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:45:03.480995+0000","last_change":"2024-05-03T18:44:46.400282+0000","last_active":"2024-05-03T18:45:03.480995+0000","last_peered":"2024-05-03T18:45:03.480995+0000","last_clean":"2024-05-03T18:45:03.480995+0000","last_became_active":"2024-05-03T18:44:46.399873+0000","last_became_peered":"2024-05-03T18:44:46.399873+0000","last_unstale":"2024-05-03T18:45:03.480995+0000","last_undegraded":"2024-05-03T18:45:03.480995+0000","last_fullsized":"2024-05-03T18:45:03.480995+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_clean_scrub_stamp":"2024-05-03T18:44:45.380544+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:12:10.005145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:44:51.430037+0000","last_change":"2024-05-03T18:44:49.422311+0000","last_active":"2024-05-03T18:44:51.430037+0000","last_peered":"2024-05-03T18:44:51.430037+0000","last_clean":"2024-05-03T18:44:51.430037+0000","last_became_active":"2024-05-03T18:44:49.421962+0000","last_became_peered":"2024-05-03T18:44:49.421962+0000","last_unstale":"2024-05-03T18:44:51.430037+0000","last_undegraded":"2024-05-03T18:44:51.430037+0000","last_fullsized":"2024-05-03T18:44:51.430037+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:50:46.407989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"78'1","reported_seq":22,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:44:51.422507+0000","last_change":"2024-05-03T18:44:46.410969+0000","last_active":"2024-05-03T18:44:51.422507+0000","last_peered":"2024-05-03T18:44:51.422507+0000","last_clean":"2024-05-03T18:44:51.422507+0000","last_became_active":"2024-05-03T18:44:46.410089+0000","last_became_peered":"2024-05-03T18:44:46.410089+0000","last_unstale":"2024-05-03T18:44:51.422507+0000","last_undegraded":"2024-05-03T18:44:51.422507+0000","last_fullsized":"2024-05-03T18:44:51.422507+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_clean_scrub_stamp":"2024-05-03T18:44:45.380544+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:13:51.641235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:44:51.431355+0000","last_change":"2024-05-03T18:44:49.426946+0000","last_active":"2024-05-03T18:44:51.431355+0000","last_peered":"2024-05-03T18:44:51.431355+0000","last_clean":"2024-05-03T18:44:51.431355+0000","last_became_active":"2024-05-03T18:44:49.426609+0000","last_became_peered":"2024-05-03T18:44:49.426609+0000","last_unstale":"2024-05-03T18:44:51.431355+0000","last_undegraded":"2024-05-03T18:44:51.431355+0000","last_fullsized":"2024-05-03T18:44:51.431355+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:34:59.944769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"78'3","reported_seq":46,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:45:25.522084+0000","last_change":"2024-05-03T18:44:46.405415+0000","last_active":"2024-05-03T18:45:25.522084+0000","last_peered":"2024-05-03T18:45:25.522084+0000","last_clean":"2024-05-03T18:45:25.522084+0000","last_became_active":"2024-05-03T18:44:46.404928+0000","last_became_peered":"2024-05-03T18:44:46.404928+0000","last_unstale":"2024-05-03T18:45:25.522084+0000","last_undegraded":"2024-05-03T18:45:25.522084+0000","last_fullsized":"2024-05-03T18:45:25.522084+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_clean_scrub_stamp":"2024-05-03T18:44:45.380544+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:33:10.979432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":22,"num_read_kb":22,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:44:51.434912+0000","last_change":"2024-05-03T18:44:49.427873+0000","last_active":"2024-05-03T18:44:51.434912+0000","last_peered":"2024-05-03T18:44:51.434912+0000","last_clean":"2024-05-03T18:44:51.434912+0000","last_became_active":"2024-05-03T18:44:49.426797+0000","last_became_peered":"2024-05-03T18:44:49.426797+0000","last_unstale":"2024-05-03T18:44:51.434912+0000","last_undegraded":"2024-05-03T18:44:51.434912+0000","last_fullsized":"2024-05-03T18:44:51.434912+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:07:28.989454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"78'2","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:45:02.477284+0000","last_change":"2024-05-03T18:44:46.407167+0000","last_active":"2024-05-03T18:45:02.477284+0000","last_peered":"2024-05-03T18:45:02.477284+0000","last_clean":"2024-05-03T18:45:02.477284+0000","last_became_active":"2024-05-03T18:44:46.406787+0000","last_became_peered":"2024-05-03T18:44:46.406787+0000","last_unstale":"2024-05-03T18:45:02.477284+0000","last_undegraded":"2024-05-03T18:45:02.477284+0000","last_fullsized":"2024-05-03T18:45:02.477284+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_clean_scrub_stamp":"2024-05-03T18:44:45.380544+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:33:12.854056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:44:51.432715+0000","last_change":"2024-05-03T18:44:49.436339+0000","last_active":"2024-05-03T18:44:51.432715+0000","last_peered":"2024-05-03T18:44:51.432715+0000","last_clean":"2024-05-03T18:44:51.432715+0000","last_became_active":"2024-05-03T18:44:49.436029+0000","last_became_peered":"2024-05-03T18:44:49.436029+0000","last_unstale":"2024-05-03T18:44:51.432715+0000","last_undegraded":"2024-05-03T18:44:51.432715+0000","last_fullsized":"2024-05-03T18:44:51.432715+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:53:47.543701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"78'4","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:45:02.476035+0000","last_change":"2024-05-03T18:44:46.647273+0000","last_active":"2024-05-03T18:45:02.476035+0000","last_peered":"2024-05-03T18:45:02.476035+0000","last_clean":"2024-05-03T18:45:02.476035+0000","last_became_active":"2024-05-03T18:44:46.646881+0000","last_became_peered":"2024-05-03T18:44:46.646881+0000","last_unstale":"2024-05-03T18:45:02.476035+0000","last_undegraded":"2024-05-03T18:45:02.476035+0000","last_fullsized":"2024-05-03T18:45:02.476035+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_clean_scrub_stamp":"2024-05-03T18:44:45.380544+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:03:47.988180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:44:51.434279+0000","last_change":"2024-05-03T18:44:49.436525+0000","last_active":"2024-05-03T18:44:51.434279+0000","last_peered":"2024-05-03T18:44:51.434279+0000","last_clean":"2024-05-03T18:44:51.434279+0000","last_became_active":"2024-05-03T18:44:49.436109+0000","last_became_peered":"2024-05-03T18:44:49.436109+0000","last_unstale":"2024-05-03T18:44:51.434279+0000","last_undegraded":"2024-05-03T18:44:51.434279+0000","last_fullsized":"2024-05-03T18:44:51.434279+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:08:42.239248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"78'25","reported_seq":97,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:45:26.523868+0000","last_change":"2024-05-03T18:44:46.407936+0000","last_active":"2024-05-03T18:45:26.523868+0000","last_peered":"2024-05-03T18:45:26.523868+0000","last_clean":"2024-05-03T18:45:26.523868+0000","last_became_active":"2024-05-03T18:44:46.407504+0000","last_became_peered":"2024-05-03T18:44:46.407504+0000","last_unstale":"2024-05-03T18:45:26.523868+0000","last_undegraded":"2024-05-03T18:45:26.523868+0000","last_fullsized":"2024-05-03T18:45:26.523868+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_clean_scrub_stamp":"2024-05-03T18:44:45.380544+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:26:27.213572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":50695,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":51,"num_read_kb":74,"num_write":26,"num_write_kb":57,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:44:51.435119+0000","last_change":"2024-05-03T18:44:49.438509+0000","last_active":"2024-05-03T18:44:51.435119+0000","last_peered":"2024-05-03T18:44:51.435119+0000","last_clean":"2024-05-03T18:44:51.435119+0000","last_became_active":"2024-05-03T18:44:49.438185+0000","last_became_peered":"2024-05-03T18:44:49.438185+0000","last_unstale":"2024-05-03T18:44:51.435119+0000","last_undegraded":"2024-05-03T18:44:51.435119+0000","last_fullsized":"2024-05-03T18:44:51.435119+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:57:28.072095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"78'6","reported_seq":29,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:45:03.483761+0000","last_change":"2024-05-03T18:44:46.407842+0000","last_active":"2024-05-03T18:45:03.483761+0000","last_peered":"2024-05-03T18:45:03.483761+0000","last_clean":"2024-05-03T18:45:03.483761+0000","last_became_active":"2024-05-03T18:44:46.407506+0000","last_became_peered":"2024-05-03T18:44:46.407506+0000","last_unstale":"2024-05-03T18:45:03.483761+0000","last_undegraded":"2024-05-03T18:45:03.483761+0000","last_fullsized":"2024-05-03T18:45:03.483761+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_clean_scrub_stamp":"2024-05-03T18:44:45.380544+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:51:33.101539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:44:51.421973+0000","last_change":"2024-05-03T18:44:49.433214+0000","last_active":"2024-05-03T18:44:51.421973+0000","last_peered":"2024-05-03T18:44:51.421973+0000","last_clean":"2024-05-03T18:44:51.421973+0000","last_became_active":"2024-05-03T18:44:49.432879+0000","last_became_peered":"2024-05-03T18:44:49.432879+0000","last_unstale":"2024-05-03T18:44:51.421973+0000","last_undegraded":"2024-05-03T18:44:51.421973+0000","last_fullsized":"2024-05-03T18:44:51.421973+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:35:17.920304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"78'3","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:45:02.475561+0000","last_change":"2024-05-03T18:44:46.405280+0000","last_active":"2024-05-03T18:45:02.475561+0000","last_peered":"2024-05-03T18:45:02.475561+0000","last_clean":"2024-05-03T18:45:02.475561+0000","last_became_active":"2024-05-03T18:44:46.404126+0000","last_became_peered":"2024-05-03T18:44:46.404126+0000","last_unstale":"2024-05-03T18:45:02.475561+0000","last_undegraded":"2024-05-03T18:45:02.475561+0000","last_fullsized":"2024-05-03T18:45:02.475561+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_clean_scrub_stamp":"2024-05-03T18:44:45.380544+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:14:52.146440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:44:51.430134+0000","last_change":"2024-05-03T18:44:49.430582+0000","last_active":"2024-05-03T18:44:51.430134+0000","last_peered":"2024-05-03T18:44:51.430134+0000","last_clean":"2024-05-03T18:44:51.430134+0000","last_became_active":"2024-05-03T18:44:49.430246+0000","last_became_peered":"2024-05-03T18:44:49.430246+0000","last_unstale":"2024-05-03T18:44:51.430134+0000","last_undegraded":"2024-05-03T18:44:51.430134+0000","last_fullsized":"2024-05-03T18:44:51.430134+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:17:13.504586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"78'3","reported_seq":49,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:45:26.523780+0000","last_change":"2024-05-03T18:44:46.404276+0000","last_active":"2024-05-03T18:45:26.523780+0000","last_peered":"2024-05-03T18:45:26.523780+0000","last_clean":"2024-05-03T18:45:26.523780+0000","last_became_active":"2024-05-03T18:44:46.403843+0000","last_became_peered":"2024-05-03T18:44:46.403843+0000","last_unstale":"2024-05-03T18:45:26.523780+0000","last_undegraded":"2024-05-03T18:45:26.523780+0000","last_fullsized":"2024-05-03T18:45:26.523780+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_clean_scrub_stamp":"2024-05-03T18:44:45.380544+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:05:49.483444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":26,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:44:51.429449+0000","last_change":"2024-05-03T18:44:49.427950+0000","last_active":"2024-05-03T18:44:51.429449+0000","last_peered":"2024-05-03T18:44:51.429449+0000","last_clean":"2024-05-03T18:44:51.429449+0000","last_became_active":"2024-05-03T18:44:49.427469+0000","last_became_peered":"2024-05-03T18:44:49.427469+0000","last_unstale":"2024-05-03T18:44:51.429449+0000","last_undegraded":"2024-05-03T18:44:51.429449+0000","last_fullsized":"2024-05-03T18:44:51.429449+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:33:35.690809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"78'15","reported_seq":37,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:45:03.484759+0000","last_change":"2024-05-03T18:44:46.645189+0000","last_active":"2024-05-03T18:45:03.484759+0000","last_peered":"2024-05-03T18:45:03.484759+0000","last_clean":"2024-05-03T18:45:03.484759+0000","last_became_active":"2024-05-03T18:44:46.644642+0000","last_became_peered":"2024-05-03T18:44:46.644642+0000","last_unstale":"2024-05-03T18:45:03.484759+0000","last_undegraded":"2024-05-03T18:45:03.484759+0000","last_fullsized":"2024-05-03T18:45:03.484759+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_clean_scrub_stamp":"2024-05-03T18:44:45.380544+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:16:49.003568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:44:51.422530+0000","last_change":"2024-05-03T18:44:49.438931+0000","last_active":"2024-05-03T18:44:51.422530+0000","last_peered":"2024-05-03T18:44:51.422530+0000","last_clean":"2024-05-03T18:44:51.422530+0000","last_became_active":"2024-05-03T18:44:49.438573+0000","last_became_peered":"2024-05-03T18:44:49.438573+0000","last_unstale":"2024-05-03T18:44:51.422530+0000","last_undegraded":"2024-05-03T18:44:51.422530+0000","last_fullsized":"2024-05-03T18:44:51.422530+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T06:09:31.042649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"78'2","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:45:05.477902+0000","last_change":"2024-05-03T18:44:46.406752+0000","last_active":"2024-05-03T18:45:05.477902+0000","last_peered":"2024-05-03T18:45:05.477902+0000","last_clean":"2024-05-03T18:45:05.477902+0000","last_became_active":"2024-05-03T18:44:46.406288+0000","last_became_peered":"2024-05-03T18:44:46.406288+0000","last_unstale":"2024-05-03T18:45:05.477902+0000","last_undegraded":"2024-05-03T18:45:05.477902+0000","last_fullsized":"2024-05-03T18:45:05.477902+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_clean_scrub_stamp":"2024-05-03T18:44:45.380544+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:20:33.947006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:44:51.429712+0000","last_change":"2024-05-03T18:44:49.431103+0000","last_active":"2024-05-03T18:44:51.429712+0000","last_peered":"2024-05-03T18:44:51.429712+0000","last_clean":"2024-05-03T18:44:51.429712+0000","last_became_active":"2024-05-03T18:44:49.430776+0000","last_became_peered":"2024-05-03T18:44:49.430776+0000","last_unstale":"2024-05-03T18:44:51.429712+0000","last_undegraded":"2024-05-03T18:44:51.429712+0000","last_fullsized":"2024-05-03T18:44:51.429712+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:40:26.357553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":21,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:44:51.431245+0000","last_change":"2024-05-03T18:44:46.647155+0000","last_active":"2024-05-03T18:44:51.431245+0000","last_peered":"2024-05-03T18:44:51.431245+0000","last_clean":"2024-05-03T18:44:51.431245+0000","last_became_active":"2024-05-03T18:44:46.646719+0000","last_became_peered":"2024-05-03T18:44:46.646719+0000","last_unstale":"2024-05-03T18:44:51.431245+0000","last_undegraded":"2024-05-03T18:44:51.431245+0000","last_fullsized":"2024-05-03T18:44:51.431245+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_clean_scrub_stamp":"2024-05-03T18:44:45.380544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T18:49:13.701498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"78'1","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:44:52.425446+0000","last_change":"2024-05-03T18:44:49.423776+0000","last_active":"2024-05-03T18:44:52.425446+0000","last_peered":"2024-05-03T18:44:52.425446+0000","last_clean":"2024-05-03T18:44:52.425446+0000","last_became_active":"2024-05-03T18:44:49.423414+0000","last_became_peered":"2024-05-03T18:44:49.423414+0000","last_unstale":"2024-05-03T18:44:52.425446+0000","last_undegraded":"2024-05-03T18:44:52.425446+0000","last_fullsized":"2024-05-03T18:44:52.425446+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:48:03.556270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"78'1","reported_seq":23,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:44:51.430304+0000","last_change":"2024-05-03T18:44:46.407725+0000","last_active":"2024-05-03T18:44:51.430304+0000","last_peered":"2024-05-03T18:44:51.430304+0000","last_clean":"2024-05-03T18:44:51.430304+0000","last_became_active":"2024-05-03T18:44:46.407317+0000","last_became_peered":"2024-05-03T18:44:46.407317+0000","last_unstale":"2024-05-03T18:44:51.430304+0000","last_undegraded":"2024-05-03T18:44:51.430304+0000","last_fullsized":"2024-05-03T18:44:51.430304+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_clean_scrub_stamp":"2024-05-03T18:44:45.380544+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:02:32.739918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:44:51.430818+0000","last_change":"2024-05-03T18:44:49.432439+0000","last_active":"2024-05-03T18:44:51.430818+0000","last_peered":"2024-05-03T18:44:51.430818+0000","last_clean":"2024-05-03T18:44:51.430818+0000","last_became_active":"2024-05-03T18:44:49.431792+0000","last_became_peered":"2024-05-03T18:44:49.431792+0000","last_unstale":"2024-05-03T18:44:51.430818+0000","last_undegraded":"2024-05-03T18:44:51.430818+0000","last_fullsized":"2024-05-03T18:44:51.430818+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:12:46.223514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:44:51.422854+0000","last_change":"2024-05-03T18:44:49.440075+0000","last_active":"2024-05-03T18:44:51.422854+0000","last_peered":"2024-05-03T18:44:51.422854+0000","last_clean":"2024-05-03T18:44:51.422854+0000","last_became_active":"2024-05-03T18:44:49.439655+0000","last_became_peered":"2024-05-03T18:44:49.439655+0000","last_unstale":"2024-05-03T18:44:51.422854+0000","last_undegraded":"2024-05-03T18:44:51.422854+0000","last_fullsized":"2024-05-03T18:44:51.422854+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:19:03.306588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"78'4","reported_seq":46,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:45:23.518594+0000","last_change":"2024-05-03T18:44:46.645090+0000","last_active":"2024-05-03T18:45:23.518594+0000","last_peered":"2024-05-03T18:45:23.518594+0000","last_clean":"2024-05-03T18:45:23.518594+0000","last_became_active":"2024-05-03T18:44:46.644722+0000","last_became_peered":"2024-05-03T18:44:46.644722+0000","last_unstale":"2024-05-03T18:45:23.518594+0000","last_undegraded":"2024-05-03T18:45:23.518594+0000","last_fullsized":"2024-05-03T18:45:23.518594+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_clean_scrub_stamp":"2024-05-03T18:44:45.380544+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:44:38.032522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:44:51.430587+0000","last_change":"2024-05-03T18:44:49.432194+0000","last_active":"2024-05-03T18:44:51.430587+0000","last_peered":"2024-05-03T18:44:51.430587+0000","last_clean":"2024-05-03T18:44:51.430587+0000","last_became_active":"2024-05-03T18:44:49.431561+0000","last_became_peered":"2024-05-03T18:44:49.431561+0000","last_unstale":"2024-05-03T18:44:51.430587+0000","last_undegraded":"2024-05-03T18:44:51.430587+0000","last_fullsized":"2024-05-03T18:44:51.430587+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:30:41.506789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"78'3","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:45:05.490231+0000","last_change":"2024-05-03T18:44:46.402866+0000","last_active":"2024-05-03T18:45:05.490231+0000","last_peered":"2024-05-03T18:45:05.490231+0000","last_clean":"2024-05-03T18:45:05.490231+0000","last_became_active":"2024-05-03T18:44:46.402394+0000","last_became_peered":"2024-05-03T18:44:46.402394+0000","last_unstale":"2024-05-03T18:45:05.490231+0000","last_undegraded":"2024-05-03T18:45:05.490231+0000","last_fullsized":"2024-05-03T18:45:05.490231+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_clean_scrub_stamp":"2024-05-03T18:44:45.380544+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:28:37.257537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":96,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:44:51.429703+0000","last_change":"2024-05-03T18:43:28.584846+0000","last_active":"2024-05-03T18:44:51.429703+0000","last_peered":"2024-05-03T18:44:51.429703+0000","last_clean":"2024-05-03T18:44:51.429703+0000","last_became_active":"2024-05-03T18:43:28.580087+0000","last_became_peered":"2024-05-03T18:43:28.580087+0000","last_unstale":"2024-05-03T18:44:51.429703+0000","last_undegraded":"2024-05-03T18:44:51.429703+0000","last_fullsized":"2024-05-03T18:44:51.429703+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:40:08.599278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:40:08.599278+0000","last_clean_scrub_stamp":"2024-05-03T18:40:08.599278+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:14:31.836127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:44:51.421924+0000","last_change":"2024-05-03T18:44:49.438545+0000","last_active":"2024-05-03T18:44:51.421924+0000","last_peered":"2024-05-03T18:44:51.421924+0000","last_clean":"2024-05-03T18:44:51.421924+0000","last_became_active":"2024-05-03T18:44:49.438178+0000","last_became_peered":"2024-05-03T18:44:49.438178+0000","last_unstale":"2024-05-03T18:44:51.421924+0000","last_undegraded":"2024-05-03T18:44:51.421924+0000","last_fullsized":"2024-05-03T18:44:51.421924+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:55:10.320262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"78'3","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:45:03.482015+0000","last_change":"2024-05-03T18:44:46.409567+0000","last_active":"2024-05-03T18:45:03.482015+0000","last_peered":"2024-05-03T18:45:03.482015+0000","last_clean":"2024-05-03T18:45:03.482015+0000","last_became_active":"2024-05-03T18:44:46.409078+0000","last_became_peered":"2024-05-03T18:44:46.409078+0000","last_unstale":"2024-05-03T18:45:03.482015+0000","last_undegraded":"2024-05-03T18:45:03.482015+0000","last_fullsized":"2024-05-03T18:45:03.482015+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_clean_scrub_stamp":"2024-05-03T18:44:45.380544+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:51:45.865890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"78'1","reported_seq":23,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:45:02.475533+0000","last_change":"2024-05-03T18:44:46.645061+0000","last_active":"2024-05-03T18:45:02.475533+0000","last_peered":"2024-05-03T18:45:02.475533+0000","last_clean":"2024-05-03T18:45:02.475533+0000","last_became_active":"2024-05-03T18:44:46.644631+0000","last_became_peered":"2024-05-03T18:44:46.644631+0000","last_unstale":"2024-05-03T18:45:02.475533+0000","last_undegraded":"2024-05-03T18:45:02.475533+0000","last_fullsized":"2024-05-03T18:45:02.475533+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_clean_scrub_stamp":"2024-05-03T18:44:45.380544+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:19:37.285434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:44:51.422352+0000","last_change":"2024-05-03T18:44:49.435674+0000","last_active":"2024-05-03T18:44:51.422352+0000","last_peered":"2024-05-03T18:44:51.422352+0000","last_clean":"2024-05-03T18:44:51.422352+0000","last_became_active":"2024-05-03T18:44:49.435333+0000","last_became_peered":"2024-05-03T18:44:49.435333+0000","last_unstale":"2024-05-03T18:44:51.422352+0000","last_undegraded":"2024-05-03T18:44:51.422352+0000","last_fullsized":"2024-05-03T18:44:51.422352+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:05:20.283773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"78'10","reported_seq":31,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:45:23.431440+0000","last_change":"2024-05-03T18:44:46.402696+0000","last_active":"2024-05-03T18:45:23.431440+0000","last_peered":"2024-05-03T18:45:23.431440+0000","last_clean":"2024-05-03T18:45:23.431440+0000","last_became_active":"2024-05-03T18:44:46.402288+0000","last_became_peered":"2024-05-03T18:44:46.402288+0000","last_unstale":"2024-05-03T18:45:23.431440+0000","last_undegraded":"2024-05-03T18:45:23.431440+0000","last_fullsized":"2024-05-03T18:45:23.431440+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_clean_scrub_stamp":"2024-05-03T18:44:45.380544+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:45:32.200586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:44:51.433587+0000","last_change":"2024-05-03T18:44:49.422754+0000","last_active":"2024-05-03T18:44:51.433587+0000","last_peered":"2024-05-03T18:44:51.433587+0000","last_clean":"2024-05-03T18:44:51.433587+0000","last_became_active":"2024-05-03T18:44:49.422385+0000","last_became_peered":"2024-05-03T18:44:49.422385+0000","last_unstale":"2024-05-03T18:44:51.433587+0000","last_undegraded":"2024-05-03T18:44:51.433587+0000","last_fullsized":"2024-05-03T18:44:51.433587+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:07:35.065842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"78'4","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:45:02.474886+0000","last_change":"2024-05-03T18:44:47.135851+0000","last_active":"2024-05-03T18:45:02.474886+0000","last_peered":"2024-05-03T18:45:02.474886+0000","last_clean":"2024-05-03T18:45:02.474886+0000","last_became_active":"2024-05-03T18:44:47.135480+0000","last_became_peered":"2024-05-03T18:44:47.135480+0000","last_unstale":"2024-05-03T18:45:02.474886+0000","last_undegraded":"2024-05-03T18:45:02.474886+0000","last_fullsized":"2024-05-03T18:45:02.474886+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_clean_scrub_stamp":"2024-05-03T18:44:45.380544+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:50:55.582018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:44:51.432400+0000","last_change":"2024-05-03T18:44:49.433123+0000","last_active":"2024-05-03T18:44:51.432400+0000","last_peered":"2024-05-03T18:44:51.432400+0000","last_clean":"2024-05-03T18:44:51.432400+0000","last_became_active":"2024-05-03T18:44:49.432781+0000","last_became_peered":"2024-05-03T18:44:49.432781+0000","last_unstale":"2024-05-03T18:44:51.432400+0000","last_undegraded":"2024-05-03T18:44:51.432400+0000","last_fullsized":"2024-05-03T18:44:51.432400+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:30:49.781052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"78'2","reported_seq":23,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:45:02.477286+0000","last_change":"2024-05-03T18:44:46.645267+0000","last_active":"2024-05-03T18:45:02.477286+0000","last_peered":"2024-05-03T18:45:02.477286+0000","last_clean":"2024-05-03T18:45:02.477286+0000","last_became_active":"2024-05-03T18:44:46.644804+0000","last_became_peered":"2024-05-03T18:44:46.644804+0000","last_unstale":"2024-05-03T18:45:02.477286+0000","last_undegraded":"2024-05-03T18:45:02.477286+0000","last_fullsized":"2024-05-03T18:45:02.477286+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_clean_scrub_stamp":"2024-05-03T18:44:45.380544+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:10:59.262218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:44:51.432318+0000","last_change":"2024-05-03T18:44:49.439352+0000","last_active":"2024-05-03T18:44:51.432318+0000","last_peered":"2024-05-03T18:44:51.432318+0000","last_clean":"2024-05-03T18:44:51.432318+0000","last_became_active":"2024-05-03T18:44:49.438981+0000","last_became_peered":"2024-05-03T18:44:49.438981+0000","last_unstale":"2024-05-03T18:44:51.432318+0000","last_undegraded":"2024-05-03T18:44:51.432318+0000","last_fullsized":"2024-05-03T18:44:51.432318+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T06:04:25.566153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"78'3","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:45:02.476275+0000","last_change":"2024-05-03T18:44:46.402305+0000","last_active":"2024-05-03T18:45:02.476275+0000","last_peered":"2024-05-03T18:45:02.476275+0000","last_clean":"2024-05-03T18:45:02.476275+0000","last_became_active":"2024-05-03T18:44:46.401633+0000","last_became_peered":"2024-05-03T18:44:46.401633+0000","last_unstale":"2024-05-03T18:45:02.476275+0000","last_undegraded":"2024-05-03T18:45:02.476275+0000","last_fullsized":"2024-05-03T18:45:02.476275+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_clean_scrub_stamp":"2024-05-03T18:44:45.380544+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:55:06.025508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:44:51.429425+0000","last_change":"2024-05-03T18:44:49.436728+0000","last_active":"2024-05-03T18:44:51.429425+0000","last_peered":"2024-05-03T18:44:51.429425+0000","last_clean":"2024-05-03T18:44:51.429425+0000","last_became_active":"2024-05-03T18:44:49.436356+0000","last_became_peered":"2024-05-03T18:44:49.436356+0000","last_unstale":"2024-05-03T18:44:51.429425+0000","last_undegraded":"2024-05-03T18:44:51.429425+0000","last_fullsized":"2024-05-03T18:44:51.429425+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:31:34.827237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"78'1","reported_seq":23,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:44:51.433245+0000","last_change":"2024-05-03T18:44:46.404949+0000","last_active":"2024-05-03T18:44:51.433245+0000","last_peered":"2024-05-03T18:44:51.433245+0000","last_clean":"2024-05-03T18:44:51.433245+0000","last_became_active":"2024-05-03T18:44:46.404516+0000","last_became_peered":"2024-05-03T18:44:46.404516+0000","last_unstale":"2024-05-03T18:44:51.433245+0000","last_undegraded":"2024-05-03T18:44:51.433245+0000","last_fullsized":"2024-05-03T18:44:51.433245+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_clean_scrub_stamp":"2024-05-03T18:44:45.380544+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:02:16.070763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:44:51.430622+0000","last_change":"2024-05-03T18:44:49.424090+0000","last_active":"2024-05-03T18:44:51.430622+0000","last_peered":"2024-05-03T18:44:51.430622+0000","last_clean":"2024-05-03T18:44:51.430622+0000","last_became_active":"2024-05-03T18:44:49.423110+0000","last_became_peered":"2024-05-03T18:44:49.423110+0000","last_unstale":"2024-05-03T18:44:51.430622+0000","last_undegraded":"2024-05-03T18:44:51.430622+0000","last_fullsized":"2024-05-03T18:44:51.430622+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:27:19.600924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"78'4","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:45:02.478333+0000","last_change":"2024-05-03T18:44:46.410438+0000","last_active":"2024-05-03T18:45:02.478333+0000","last_peered":"2024-05-03T18:45:02.478333+0000","last_clean":"2024-05-03T18:45:02.478333+0000","last_became_active":"2024-05-03T18:44:46.410061+0000","last_became_peered":"2024-05-03T18:44:46.410061+0000","last_unstale":"2024-05-03T18:45:02.478333+0000","last_undegraded":"2024-05-03T18:45:02.478333+0000","last_fullsized":"2024-05-03T18:45:02.478333+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_clean_scrub_stamp":"2024-05-03T18:44:45.380544+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:30:57.668522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:44:51.430303+0000","last_change":"2024-05-03T18:44:49.432498+0000","last_active":"2024-05-03T18:44:51.430303+0000","last_peered":"2024-05-03T18:44:51.430303+0000","last_clean":"2024-05-03T18:44:51.430303+0000","last_became_active":"2024-05-03T18:44:49.432152+0000","last_became_peered":"2024-05-03T18:44:49.432152+0000","last_unstale":"2024-05-03T18:44:51.430303+0000","last_undegraded":"2024-05-03T18:44:51.430303+0000","last_fullsized":"2024-05-03T18:44:51.430303+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:07:35.058213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"78'3","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:45:03.483725+0000","last_change":"2024-05-03T18:44:47.135973+0000","last_active":"2024-05-03T18:45:03.483725+0000","last_peered":"2024-05-03T18:45:03.483725+0000","last_clean":"2024-05-03T18:45:03.483725+0000","last_became_active":"2024-05-03T18:44:47.135548+0000","last_became_peered":"2024-05-03T18:44:47.135548+0000","last_unstale":"2024-05-03T18:45:03.483725+0000","last_undegraded":"2024-05-03T18:45:03.483725+0000","last_fullsized":"2024-05-03T18:45:03.483725+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_clean_scrub_stamp":"2024-05-03T18:44:45.380544+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:06:42.858462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:44:51.430484+0000","last_change":"2024-05-03T18:44:49.432722+0000","last_active":"2024-05-03T18:44:51.430484+0000","last_peered":"2024-05-03T18:44:51.430484+0000","last_clean":"2024-05-03T18:44:51.430484+0000","last_became_active":"2024-05-03T18:44:49.432393+0000","last_became_peered":"2024-05-03T18:44:49.432393+0000","last_unstale":"2024-05-03T18:44:51.430484+0000","last_undegraded":"2024-05-03T18:44:51.430484+0000","last_fullsized":"2024-05-03T18:44:51.430484+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:44:20.551442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"78'1","reported_seq":23,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:44:51.433380+0000","last_change":"2024-05-03T18:44:46.402489+0000","last_active":"2024-05-03T18:44:51.433380+0000","last_peered":"2024-05-03T18:44:51.433380+0000","last_clean":"2024-05-03T18:44:51.433380+0000","last_became_active":"2024-05-03T18:44:46.401992+0000","last_became_peered":"2024-05-03T18:44:46.401992+0000","last_unstale":"2024-05-03T18:44:51.433380+0000","last_undegraded":"2024-05-03T18:44:51.433380+0000","last_fullsized":"2024-05-03T18:44:51.433380+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_clean_scrub_stamp":"2024-05-03T18:44:45.380544+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:09:10.600254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"78'1","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:44:51.430233+0000","last_change":"2024-05-03T18:44:49.430095+0000","last_active":"2024-05-03T18:44:51.430233+0000","last_peered":"2024-05-03T18:44:51.430233+0000","last_clean":"2024-05-03T18:44:51.430233+0000","last_became_active":"2024-05-03T18:44:49.429590+0000","last_became_peered":"2024-05-03T18:44:49.429590+0000","last_unstale":"2024-05-03T18:44:51.430233+0000","last_undegraded":"2024-05-03T18:44:51.430233+0000","last_fullsized":"2024-05-03T18:44:51.430233+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:50:29.362826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"78'2","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:44:51.430328+0000","last_change":"2024-05-03T18:44:46.404414+0000","last_active":"2024-05-03T18:44:51.430328+0000","last_peered":"2024-05-03T18:44:51.430328+0000","last_clean":"2024-05-03T18:44:51.430328+0000","last_became_active":"2024-05-03T18:44:46.404136+0000","last_became_peered":"2024-05-03T18:44:46.404136+0000","last_unstale":"2024-05-03T18:44:51.430328+0000","last_undegraded":"2024-05-03T18:44:51.430328+0000","last_fullsized":"2024-05-03T18:44:51.430328+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_clean_scrub_stamp":"2024-05-03T18:44:45.380544+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:24:58.125984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:44:51.430452+0000","last_change":"2024-05-03T18:44:49.429013+0000","last_active":"2024-05-03T18:44:51.430452+0000","last_peered":"2024-05-03T18:44:51.430452+0000","last_clean":"2024-05-03T18:44:51.430452+0000","last_became_active":"2024-05-03T18:44:49.428380+0000","last_became_peered":"2024-05-03T18:44:49.428380+0000","last_unstale":"2024-05-03T18:44:51.430452+0000","last_undegraded":"2024-05-03T18:44:51.430452+0000","last_fullsized":"2024-05-03T18:44:51.430452+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:13:00.073384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:44:51.429455+0000","last_change":"2024-05-03T18:44:49.437186+0000","last_active":"2024-05-03T18:44:51.429455+0000","last_peered":"2024-05-03T18:44:51.429455+0000","last_clean":"2024-05-03T18:44:51.429455+0000","last_became_active":"2024-05-03T18:44:49.435192+0000","last_became_peered":"2024-05-03T18:44:49.435192+0000","last_unstale":"2024-05-03T18:44:51.429455+0000","last_undegraded":"2024-05-03T18:44:51.429455+0000","last_fullsized":"2024-05-03T18:44:51.429455+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:16:25.707746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:44:51.431909+0000","last_change":"2024-05-03T18:44:49.441553+0000","last_active":"2024-05-03T18:44:51.431909+0000","last_peered":"2024-05-03T18:44:51.431909+0000","last_clean":"2024-05-03T18:44:51.431909+0000","last_became_active":"2024-05-03T18:44:49.441155+0000","last_became_peered":"2024-05-03T18:44:49.441155+0000","last_unstale":"2024-05-03T18:44:51.431909+0000","last_undegraded":"2024-05-03T18:44:51.431909+0000","last_fullsized":"2024-05-03T18:44:51.431909+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:53:53.888748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:44:51.431926+0000","last_change":"2024-05-03T18:44:49.436641+0000","last_active":"2024-05-03T18:44:51.431926+0000","last_peered":"2024-05-03T18:44:51.431926+0000","last_clean":"2024-05-03T18:44:51.431926+0000","last_became_active":"2024-05-03T18:44:49.436232+0000","last_became_peered":"2024-05-03T18:44:49.436232+0000","last_unstale":"2024-05-03T18:44:51.431926+0000","last_undegraded":"2024-05-03T18:44:51.431926+0000","last_fullsized":"2024-05-03T18:44:51.431926+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:06:55.773088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:44:51.429683+0000","last_change":"2024-05-03T18:44:49.434394+0000","last_active":"2024-05-03T18:44:51.429683+0000","last_peered":"2024-05-03T18:44:51.429683+0000","last_clean":"2024-05-03T18:44:51.429683+0000","last_became_active":"2024-05-03T18:44:49.433995+0000","last_became_peered":"2024-05-03T18:44:49.433995+0000","last_unstale":"2024-05-03T18:44:51.429683+0000","last_undegraded":"2024-05-03T18:44:51.429683+0000","last_fullsized":"2024-05-03T18:44:51.429683+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:25:20.478426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:44:51.422456+0000","last_change":"2024-05-03T18:44:49.439218+0000","last_active":"2024-05-03T18:44:51.422456+0000","last_peered":"2024-05-03T18:44:51.422456+0000","last_clean":"2024-05-03T18:44:51.422456+0000","last_became_active":"2024-05-03T18:44:49.438875+0000","last_became_peered":"2024-05-03T18:44:49.438875+0000","last_unstale":"2024-05-03T18:44:51.422456+0000","last_undegraded":"2024-05-03T18:44:51.422456+0000","last_fullsized":"2024-05-03T18:44:51.422456+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:08:55.899829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:44:51.429841+0000","last_change":"2024-05-03T18:44:49.433079+0000","last_active":"2024-05-03T18:44:51.429841+0000","last_peered":"2024-05-03T18:44:51.429841+0000","last_clean":"2024-05-03T18:44:51.429841+0000","last_became_active":"2024-05-03T18:44:49.432740+0000","last_became_peered":"2024-05-03T18:44:49.432740+0000","last_unstale":"2024-05-03T18:44:51.429841+0000","last_undegraded":"2024-05-03T18:44:51.429841+0000","last_fullsized":"2024-05-03T18:44:51.429841+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:31:36.317667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:44:51.431918+0000","last_change":"2024-05-03T18:44:49.435321+0000","last_active":"2024-05-03T18:44:51.431918+0000","last_peered":"2024-05-03T18:44:51.431918+0000","last_clean":"2024-05-03T18:44:51.431918+0000","last_became_active":"2024-05-03T18:44:49.434935+0000","last_became_peered":"2024-05-03T18:44:49.434935+0000","last_unstale":"2024-05-03T18:44:51.431918+0000","last_undegraded":"2024-05-03T18:44:51.431918+0000","last_fullsized":"2024-05-03T18:44:51.431918+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:14:32.406164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:44:51.430935+0000","last_change":"2024-05-03T18:44:49.438202+0000","last_active":"2024-05-03T18:44:51.430935+0000","last_peered":"2024-05-03T18:44:51.430935+0000","last_clean":"2024-05-03T18:44:51.430935+0000","last_became_active":"2024-05-03T18:44:49.437822+0000","last_became_peered":"2024-05-03T18:44:49.437822+0000","last_unstale":"2024-05-03T18:44:51.430935+0000","last_undegraded":"2024-05-03T18:44:51.430935+0000","last_fullsized":"2024-05-03T18:44:51.430935+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:12:54.417022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:44:51.422317+0000","last_change":"2024-05-03T18:44:49.434992+0000","last_active":"2024-05-03T18:44:51.422317+0000","last_peered":"2024-05-03T18:44:51.422317+0000","last_clean":"2024-05-03T18:44:51.422317+0000","last_became_active":"2024-05-03T18:44:49.434668+0000","last_became_peered":"2024-05-03T18:44:49.434668+0000","last_unstale":"2024-05-03T18:44:51.422317+0000","last_undegraded":"2024-05-03T18:44:51.422317+0000","last_fullsized":"2024-05-03T18:44:51.422317+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:20:48.527470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:44:51.430027+0000","last_change":"2024-05-03T18:44:49.432690+0000","last_active":"2024-05-03T18:44:51.430027+0000","last_peered":"2024-05-03T18:44:51.430027+0000","last_clean":"2024-05-03T18:44:51.430027+0000","last_became_active":"2024-05-03T18:44:49.432389+0000","last_became_peered":"2024-05-03T18:44:49.432389+0000","last_unstale":"2024-05-03T18:44:51.430027+0000","last_undegraded":"2024-05-03T18:44:51.430027+0000","last_fullsized":"2024-05-03T18:44:51.430027+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:51:27.326901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:44:51.433768+0000","last_change":"2024-05-03T18:44:49.434205+0000","last_active":"2024-05-03T18:44:51.433768+0000","last_peered":"2024-05-03T18:44:51.433768+0000","last_clean":"2024-05-03T18:44:51.433768+0000","last_became_active":"2024-05-03T18:44:49.433619+0000","last_became_peered":"2024-05-03T18:44:49.433619+0000","last_unstale":"2024-05-03T18:44:51.433768+0000","last_undegraded":"2024-05-03T18:44:51.433768+0000","last_fullsized":"2024-05-03T18:44:51.433768+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:36:15.933276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:44:51.422266+0000","last_change":"2024-05-03T18:44:49.425276+0000","last_active":"2024-05-03T18:44:51.422266+0000","last_peered":"2024-05-03T18:44:51.422266+0000","last_clean":"2024-05-03T18:44:51.422266+0000","last_became_active":"2024-05-03T18:44:49.424916+0000","last_became_peered":"2024-05-03T18:44:49.424916+0000","last_unstale":"2024-05-03T18:44:51.422266+0000","last_undegraded":"2024-05-03T18:44:51.422266+0000","last_fullsized":"2024-05-03T18:44:51.422266+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:09:03.512860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:44:51.429434+0000","last_change":"2024-05-03T18:44:49.433335+0000","last_active":"2024-05-03T18:44:51.429434+0000","last_peered":"2024-05-03T18:44:51.429434+0000","last_clean":"2024-05-03T18:44:51.429434+0000","last_became_active":"2024-05-03T18:44:49.432991+0000","last_became_peered":"2024-05-03T18:44:49.432991+0000","last_unstale":"2024-05-03T18:44:51.429434+0000","last_undegraded":"2024-05-03T18:44:51.429434+0000","last_fullsized":"2024-05-03T18:44:51.429434+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:31:25.182590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":54696,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":156,"num_read_kb":135,"num_write":157,"num_write_kb":98,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":166364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":134,"ondisk_log_size":134,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710741,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27836,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717316,"statfs":{"total":95995035648,"available":95966531584,"internally_reserved":0,"allocated":839680,"data_stored":408028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906971,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28508,"kb_used_data":1492,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716644,"statfs":{"total":95995035648,"available":95965843456,"internally_reserved":0,"allocated":1527808,"data_stored":1050204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070495,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27852,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717300,"statfs":{"total":95995035648,"available":95966515200,"internally_reserved":0,"allocated":856064,"data_stored":408264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266725,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27852,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717300,"statfs":{"total":95995035648,"available":95966515200,"internally_reserved":0,"allocated":856064,"data_stored":408184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462954,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36752,"kb_used_data":1416,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708400,"statfs":{"total":95995035648,"available":95957401600,"internally_reserved":0,"allocated":1449984,"data_stored":1000114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626477,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27840,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717312,"statfs":{"total":95995035648,"available":95966527488,"internally_reserved":0,"allocated":843776,"data_stored":407653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574920,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27892,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717260,"statfs":{"total":95995035648,"available":95966474240,"internally_reserved":0,"allocated":897024,"data_stored":459279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738443,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36180,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708972,"statfs":{"total":95995035648,"available":95957987328,"internally_reserved":0,"allocated":864256,"data_stored":408924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411395,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":921600,"data_stored":461020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182462,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36176,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708976,"statfs":{"total":95995035648,"available":95957991424,"internally_reserved":0,"allocated":860160,"data_stored":407964,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986233,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27844,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717308,"statfs":{"total":95995035648,"available":95966523392,"internally_reserved":0,"allocated":847872,"data_stored":408090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822707,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36764,"kb_used_data":1428,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708388,"statfs":{"total":95995035648,"available":95957389312,"internally_reserved":0,"allocated":1462272,"data_stored":999798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":51833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":53574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":52273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-03T18:45:31.810 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid f500455e-097b-11ef-bc97-c7b262605968 -- ceph pg dump --format=json 2024-05-03T18:45:31.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:45:31 smithi045 ceph-mon[33299]: pgmap v300: 97 pgs: 97 active+clean; 630 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 2.9 KiB/s wr, 6 op/s 2024-05-03T18:45:32.061 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:45:32.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:45:32 smithi032 ceph-mon[29763]: from='client.14748 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-03T18:45:32.997 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:45:32 smithi045 ceph-mon[33299]: from='client.14748 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-03T18:45:33.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:45:32 smithi113 ceph-mon[33540]: from='client.14748 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-03T18:45:33.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:33.793+0000 7f9b60fdb700 1 -- 172.21.15.32:0/1630676971 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b5c102d60 msgr2=0x7f9b5c10f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:33.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:33.793+0000 7f9b60fdb700 1 --2- 172.21.15.32:0/1630676971 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b5c102d60 0x7f9b5c10f250 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f9b5400b0a0 tx=0x7f9b5400b3b0 comp rx=0 tx=0).stop 2024-05-03T18:45:33.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:33.793+0000 7f9b60fdb700 1 -- 172.21.15.32:0/1630676971 shutdown_connections 2024-05-03T18:45:33.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:33.793+0000 7f9b60fdb700 1 --2- 172.21.15.32:0/1630676971 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b5c102d60 0x7f9b5c10f250 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:33.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:33.793+0000 7f9b60fdb700 1 --2- 172.21.15.32:0/1630676971 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9b5c108410 0x7f9b5c1087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:33.796 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:33.793+0000 7f9b60fdb700 1 --2- 172.21.15.32:0/1630676971 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9b5c1023a0 0x7f9b5c102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:33.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:33.793+0000 7f9b60fdb700 1 -- 172.21.15.32:0/1630676971 >> 172.21.15.32:0/1630676971 conn(0x7f9b5c079350 msgr2=0x7f9b5c079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:45:33.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:33.793+0000 7f9b60fdb700 1 -- 172.21.15.32:0/1630676971 shutdown_connections 2024-05-03T18:45:33.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:33.793+0000 7f9b60fdb700 1 -- 172.21.15.32:0/1630676971 wait complete. 2024-05-03T18:45:33.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:33.794+0000 7f9b60fdb700 1 Processor -- start 2024-05-03T18:45:33.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:33.794+0000 7f9b60fdb700 1 -- start start 2024-05-03T18:45:33.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:33.794+0000 7f9b60fdb700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b5c1023a0 0x7f9b5c19f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:33.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:33.794+0000 7f9b60fdb700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9b5c102d60 0x7f9b5c19fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:33.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:33.794+0000 7f9b60fdb700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9b5c108410 0x7f9b5c199840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:33.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:33.794+0000 7f9b60fdb700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f9b5c112530 con 0x7f9b5c1023a0 2024-05-03T18:45:33.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:33.794+0000 7f9b60fdb700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f9b5c1123b0 con 0x7f9b5c108410 2024-05-03T18:45:33.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:33.794+0000 7f9b60fdb700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f9b5c1126b0 con 0x7f9b5c102d60 2024-05-03T18:45:33.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:33.794+0000 7f9b5a59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b5c1023a0 0x7f9b5c19f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:33.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:33.794+0000 7f9b5a59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b5c1023a0 0x7f9b5c19f670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38546/0 (socket says 172.21.15.32:38546) 2024-05-03T18:45:33.797 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:33.794+0000 7f9b5a59c700 1 -- 172.21.15.32:0/1052269873 learned_addr learned my addr 172.21.15.32:0/1052269873 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:45:33.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:33.795+0000 7f9b59d9b700 1 --2- 172.21.15.32:0/1052269873 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9b5c102d60 0x7f9b5c19fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:33.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:33.795+0000 7f9b5ad9d700 1 --2- 172.21.15.32:0/1052269873 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9b5c108410 0x7f9b5c199840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:33.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:33.795+0000 7f9b5a59c700 1 -- 172.21.15.32:0/1052269873 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9b5c102d60 msgr2=0x7f9b5c19fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:33.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:33.795+0000 7f9b5a59c700 1 --2- 172.21.15.32:0/1052269873 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9b5c102d60 0x7f9b5c19fbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:33.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:33.795+0000 7f9b5a59c700 1 -- 172.21.15.32:0/1052269873 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9b5c108410 msgr2=0x7f9b5c199840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:33.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:33.795+0000 7f9b5a59c700 1 --2- 172.21.15.32:0/1052269873 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9b5c108410 0x7f9b5c199840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:33.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:33.795+0000 7f9b5a59c700 1 -- 172.21.15.32:0/1052269873 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b5c19a020 con 0x7f9b5c1023a0 2024-05-03T18:45:33.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:33.795+0000 7f9b5ad9d700 1 --2- 172.21.15.32:0/1052269873 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9b5c108410 0x7f9b5c199840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:45:33.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:33.795+0000 7f9b5a59c700 1 --2- 172.21.15.32:0/1052269873 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b5c1023a0 0x7f9b5c19f670 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f9b4c00c9b0 tx=0x7f9b4c00ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:45:33.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:33.795+0000 7f9b4b7fe700 1 -- 172.21.15.32:0/1052269873 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b4c007b00 con 0x7f9b5c1023a0 2024-05-03T18:45:33.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:33.795+0000 7f9b4b7fe700 1 -- 172.21.15.32:0/1052269873 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9b4c007ca0 con 0x7f9b5c1023a0 2024-05-03T18:45:33.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:33.795+0000 7f9b60fdb700 1 -- 172.21.15.32:0/1052269873 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b5c19a310 con 0x7f9b5c1023a0 2024-05-03T18:45:33.798 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:33.795+0000 7f9b4b7fe700 1 -- 172.21.15.32:0/1052269873 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b4c018660 con 0x7f9b5c1023a0 2024-05-03T18:45:33.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:33.795+0000 7f9b60fdb700 1 -- 172.21.15.32:0/1052269873 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b5c1a64d0 con 0x7f9b5c1023a0 2024-05-03T18:45:33.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:33.796+0000 7f9b60fdb700 1 -- 172.21.15.32:0/1052269873 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b5c103ac0 con 0x7f9b5c1023a0 2024-05-03T18:45:33.799 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:33.798+0000 7f9b4b7fe700 1 -- 172.21.15.32:0/1052269873 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f9b4c00f450 con 0x7f9b5c1023a0 2024-05-03T18:45:33.799 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:45:33 smithi032 ceph-mon[29763]: pgmap v301: 97 pgs: 97 active+clean; 630 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 2.9 KiB/s wr, 7 op/s 2024-05-03T18:45:33.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:33.799+0000 7f9b4b7fe700 1 --2- 172.21.15.32:0/1052269873 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f9b440b2910 0x7f9b440b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:33.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:33.799+0000 7f9b59d9b700 1 --2- 172.21.15.32:0/1052269873 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f9b440b2910 0x7f9b440b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:33.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:33.799+0000 7f9b4b7fe700 1 -- 172.21.15.32:0/1052269873 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7979+0+0 (secure 0 0 0) 0x7f9b4c0e2400 con 0x7f9b5c1023a0 2024-05-03T18:45:33.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:33.800+0000 7f9b59d9b700 1 --2- 172.21.15.32:0/1052269873 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f9b440b2910 0x7f9b440b4dd0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f9b5c079950 tx=0x7f9b5000a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:45:33.804 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:33.803+0000 7f9b4b7fe700 1 -- 172.21.15.32:0/1052269873 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f9b4c014030 con 0x7f9b5c1023a0 2024-05-03T18:45:33.997 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:45:33 smithi045 ceph-mon[33299]: pgmap v301: 97 pgs: 97 active+clean; 630 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 2.9 KiB/s wr, 7 op/s 2024-05-03T18:45:34.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:34.013+0000 7f9b60fdb700 1 -- 172.21.15.32:0/1052269873 --> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9b5c1061c0 con 0x7f9b440b2910 2024-05-03T18:45:34.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:34.020+0000 7f9b4b7fe700 1 -- 172.21.15.32:0/1052269873 <== mgr.14152 v2:172.21.15.32:6800/1796547904 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258697 (secure 0 0 0) 0x7f9b5c1061c0 con 0x7f9b440b2910 2024-05-03T18:45:34.025 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-03T18:45:34.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:34.024+0000 7f9b60fdb700 1 -- 172.21.15.32:0/1052269873 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f9b440b2910 msgr2=0x7f9b440b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:34.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:34.024+0000 7f9b60fdb700 1 --2- 172.21.15.32:0/1052269873 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f9b440b2910 0x7f9b440b4dd0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f9b5c079950 tx=0x7f9b5000a380 comp rx=0 tx=0).stop 2024-05-03T18:45:34.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:34.024+0000 7f9b60fdb700 1 -- 172.21.15.32:0/1052269873 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b5c1023a0 msgr2=0x7f9b5c19f670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:34.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:34.024+0000 7f9b60fdb700 1 --2- 172.21.15.32:0/1052269873 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b5c1023a0 0x7f9b5c19f670 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f9b4c00c9b0 tx=0x7f9b4c00ccc0 comp rx=0 tx=0).stop 2024-05-03T18:45:34.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:34.024+0000 7f9b60fdb700 1 -- 172.21.15.32:0/1052269873 shutdown_connections 2024-05-03T18:45:34.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:34.024+0000 7f9b60fdb700 1 --2- 172.21.15.32:0/1052269873 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f9b440b2910 0x7f9b440b4dd0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:34.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:34.024+0000 7f9b60fdb700 1 --2- 172.21.15.32:0/1052269873 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9b5c108410 0x7f9b5c199840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:34.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:34.024+0000 7f9b60fdb700 1 --2- 172.21.15.32:0/1052269873 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b5c1023a0 0x7f9b5c19f670 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:34.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:34.024+0000 7f9b60fdb700 1 --2- 172.21.15.32:0/1052269873 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9b5c102d60 0x7f9b5c19fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:34.027 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:34.024+0000 7f9b60fdb700 1 -- 172.21.15.32:0/1052269873 >> 172.21.15.32:0/1052269873 conn(0x7f9b5c079350 msgr2=0x7f9b5c06b380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:45:34.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:34.024+0000 7f9b60fdb700 1 -- 172.21.15.32:0/1052269873 shutdown_connections 2024-05-03T18:45:34.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:34.024+0000 7f9b60fdb700 1 -- 172.21.15.32:0/1052269873 wait complete. 2024-05-03T18:45:34.028 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-05-03T18:45:34.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:45:33 smithi113 ceph-mon[33540]: pgmap v301: 97 pgs: 97 active+clean; 630 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 2.9 KiB/s wr, 7 op/s 2024-05-03T18:45:34.677 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":302,"stamp":"2024-05-03T18:45:33.974268+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":646436,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":263,"num_read_kb":216,"num_write":284,"num_write_kb":2253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":230,"ondisk_log_size":230,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":369412,"kb_used_data":11940,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124572412,"statfs":{"total":1151940427776,"available":1151562149888,"internally_reserved":0,"allocated":12226560,"data_stored":6828660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":365933883},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":29503,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":84,"num_read_kb":71,"num_write":15,"num_write_kb":35,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003257"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:44:51.430010+0000","last_change":"2024-05-03T18:44:49.430467+0000","last_active":"2024-05-03T18:44:51.430010+0000","last_peered":"2024-05-03T18:44:51.430010+0000","last_clean":"2024-05-03T18:44:51.430010+0000","last_became_active":"2024-05-03T18:44:49.430147+0000","last_became_peered":"2024-05-03T18:44:49.430147+0000","last_unstale":"2024-05-03T18:44:51.430010+0000","last_undegraded":"2024-05-03T18:44:51.430010+0000","last_fullsized":"2024-05-03T18:44:51.430010+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:49:22.725690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:44:51.432156+0000","last_change":"2024-05-03T18:44:49.432057+0000","last_active":"2024-05-03T18:44:51.432156+0000","last_peered":"2024-05-03T18:44:51.432156+0000","last_clean":"2024-05-03T18:44:51.432156+0000","last_became_active":"2024-05-03T18:44:49.430867+0000","last_became_peered":"2024-05-03T18:44:49.430867+0000","last_unstale":"2024-05-03T18:44:51.432156+0000","last_undegraded":"2024-05-03T18:44:51.432156+0000","last_fullsized":"2024-05-03T18:44:51.432156+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:31:16.470041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:44:51.432046+0000","last_change":"2024-05-03T18:44:49.433622+0000","last_active":"2024-05-03T18:44:51.432046+0000","last_peered":"2024-05-03T18:44:51.432046+0000","last_clean":"2024-05-03T18:44:51.432046+0000","last_became_active":"2024-05-03T18:44:49.431803+0000","last_became_peered":"2024-05-03T18:44:49.431803+0000","last_unstale":"2024-05-03T18:44:51.432046+0000","last_undegraded":"2024-05-03T18:44:51.432046+0000","last_fullsized":"2024-05-03T18:44:51.432046+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:40:18.492498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:44:51.423005+0000","last_change":"2024-05-03T18:44:49.437469+0000","last_active":"2024-05-03T18:44:51.423005+0000","last_peered":"2024-05-03T18:44:51.423005+0000","last_clean":"2024-05-03T18:44:51.423005+0000","last_became_active":"2024-05-03T18:44:49.435519+0000","last_became_peered":"2024-05-03T18:44:49.435519+0000","last_unstale":"2024-05-03T18:44:51.423005+0000","last_undegraded":"2024-05-03T18:44:51.423005+0000","last_fullsized":"2024-05-03T18:44:51.423005+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:11:07.246146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:44:51.434741+0000","last_change":"2024-05-03T18:44:49.435115+0000","last_active":"2024-05-03T18:44:51.434741+0000","last_peered":"2024-05-03T18:44:51.434741+0000","last_clean":"2024-05-03T18:44:51.434741+0000","last_became_active":"2024-05-03T18:44:49.434820+0000","last_became_peered":"2024-05-03T18:44:49.434820+0000","last_unstale":"2024-05-03T18:44:51.434741+0000","last_undegraded":"2024-05-03T18:44:51.434741+0000","last_fullsized":"2024-05-03T18:44:51.434741+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:32:24.817201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:44:51.430766+0000","last_change":"2024-05-03T18:44:49.432143+0000","last_active":"2024-05-03T18:44:51.430766+0000","last_peered":"2024-05-03T18:44:51.430766+0000","last_clean":"2024-05-03T18:44:51.430766+0000","last_became_active":"2024-05-03T18:44:49.431742+0000","last_became_peered":"2024-05-03T18:44:49.431742+0000","last_unstale":"2024-05-03T18:44:51.430766+0000","last_undegraded":"2024-05-03T18:44:51.430766+0000","last_fullsized":"2024-05-03T18:44:51.430766+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:25:36.384277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:44:51.429846+0000","last_change":"2024-05-03T18:44:49.428955+0000","last_active":"2024-05-03T18:44:51.429846+0000","last_peered":"2024-05-03T18:44:51.429846+0000","last_clean":"2024-05-03T18:44:51.429846+0000","last_became_active":"2024-05-03T18:44:49.428587+0000","last_became_peered":"2024-05-03T18:44:49.428587+0000","last_unstale":"2024-05-03T18:44:51.429846+0000","last_undegraded":"2024-05-03T18:44:51.429846+0000","last_fullsized":"2024-05-03T18:44:51.429846+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:14:34.637035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:44:51.430027+0000","last_change":"2024-05-03T18:44:49.432804+0000","last_active":"2024-05-03T18:44:51.430027+0000","last_peered":"2024-05-03T18:44:51.430027+0000","last_clean":"2024-05-03T18:44:51.430027+0000","last_became_active":"2024-05-03T18:44:49.431828+0000","last_became_peered":"2024-05-03T18:44:49.431828+0000","last_unstale":"2024-05-03T18:44:51.430027+0000","last_undegraded":"2024-05-03T18:44:51.430027+0000","last_fullsized":"2024-05-03T18:44:51.430027+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:06:40.810972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:44:51.435402+0000","last_change":"2024-05-03T18:44:49.428919+0000","last_active":"2024-05-03T18:44:51.435402+0000","last_peered":"2024-05-03T18:44:51.435402+0000","last_clean":"2024-05-03T18:44:51.435402+0000","last_became_active":"2024-05-03T18:44:49.428350+0000","last_became_peered":"2024-05-03T18:44:49.428350+0000","last_unstale":"2024-05-03T18:44:51.435402+0000","last_undegraded":"2024-05-03T18:44:51.435402+0000","last_fullsized":"2024-05-03T18:44:51.435402+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T18:50:09.895485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:44:51.431609+0000","last_change":"2024-05-03T18:44:49.431765+0000","last_active":"2024-05-03T18:44:51.431609+0000","last_peered":"2024-05-03T18:44:51.431609+0000","last_clean":"2024-05-03T18:44:51.431609+0000","last_became_active":"2024-05-03T18:44:49.431417+0000","last_became_peered":"2024-05-03T18:44:49.431417+0000","last_unstale":"2024-05-03T18:44:51.431609+0000","last_undegraded":"2024-05-03T18:44:51.431609+0000","last_fullsized":"2024-05-03T18:44:51.431609+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:24:58.097954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:44:51.430506+0000","last_change":"2024-05-03T18:44:49.434154+0000","last_active":"2024-05-03T18:44:51.430506+0000","last_peered":"2024-05-03T18:44:51.430506+0000","last_clean":"2024-05-03T18:44:51.430506+0000","last_became_active":"2024-05-03T18:44:49.433806+0000","last_became_peered":"2024-05-03T18:44:49.433806+0000","last_unstale":"2024-05-03T18:44:51.430506+0000","last_undegraded":"2024-05-03T18:44:51.430506+0000","last_fullsized":"2024-05-03T18:44:51.430506+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:52:39.233556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:44:51.422984+0000","last_change":"2024-05-03T18:44:49.435385+0000","last_active":"2024-05-03T18:44:51.422984+0000","last_peered":"2024-05-03T18:44:51.422984+0000","last_clean":"2024-05-03T18:44:51.422984+0000","last_became_active":"2024-05-03T18:44:49.434727+0000","last_became_peered":"2024-05-03T18:44:49.434727+0000","last_unstale":"2024-05-03T18:44:51.422984+0000","last_undegraded":"2024-05-03T18:44:51.422984+0000","last_fullsized":"2024-05-03T18:44:51.422984+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:08:45.986371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:44:51.422322+0000","last_change":"2024-05-03T18:44:49.428536+0000","last_active":"2024-05-03T18:44:51.422322+0000","last_peered":"2024-05-03T18:44:51.422322+0000","last_clean":"2024-05-03T18:44:51.422322+0000","last_became_active":"2024-05-03T18:44:49.427630+0000","last_became_peered":"2024-05-03T18:44:49.427630+0000","last_unstale":"2024-05-03T18:44:51.422322+0000","last_undegraded":"2024-05-03T18:44:51.422322+0000","last_fullsized":"2024-05-03T18:44:51.422322+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:24:09.622403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:44:51.429974+0000","last_change":"2024-05-03T18:44:49.429236+0000","last_active":"2024-05-03T18:44:51.429974+0000","last_peered":"2024-05-03T18:44:51.429974+0000","last_clean":"2024-05-03T18:44:51.429974+0000","last_became_active":"2024-05-03T18:44:49.428942+0000","last_became_peered":"2024-05-03T18:44:49.428942+0000","last_unstale":"2024-05-03T18:44:51.429974+0000","last_undegraded":"2024-05-03T18:44:51.429974+0000","last_fullsized":"2024-05-03T18:44:51.429974+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:16:44.241736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:44:51.429575+0000","last_change":"2024-05-03T18:44:49.433388+0000","last_active":"2024-05-03T18:44:51.429575+0000","last_peered":"2024-05-03T18:44:51.429575+0000","last_clean":"2024-05-03T18:44:51.429575+0000","last_became_active":"2024-05-03T18:44:49.431869+0000","last_became_peered":"2024-05-03T18:44:49.431869+0000","last_unstale":"2024-05-03T18:44:51.429575+0000","last_undegraded":"2024-05-03T18:44:51.429575+0000","last_fullsized":"2024-05-03T18:44:51.429575+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:36:52.803391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:44:51.433507+0000","last_change":"2024-05-03T18:44:49.425781+0000","last_active":"2024-05-03T18:44:51.433507+0000","last_peered":"2024-05-03T18:44:51.433507+0000","last_clean":"2024-05-03T18:44:51.433507+0000","last_became_active":"2024-05-03T18:44:49.425248+0000","last_became_peered":"2024-05-03T18:44:49.425248+0000","last_unstale":"2024-05-03T18:44:51.433507+0000","last_undegraded":"2024-05-03T18:44:51.433507+0000","last_fullsized":"2024-05-03T18:44:51.433507+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:27:25.017459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:44:51.430174+0000","last_change":"2024-05-03T18:44:49.430099+0000","last_active":"2024-05-03T18:44:51.430174+0000","last_peered":"2024-05-03T18:44:51.430174+0000","last_clean":"2024-05-03T18:44:51.430174+0000","last_became_active":"2024-05-03T18:44:49.429685+0000","last_became_peered":"2024-05-03T18:44:49.429685+0000","last_unstale":"2024-05-03T18:44:51.430174+0000","last_undegraded":"2024-05-03T18:44:51.430174+0000","last_fullsized":"2024-05-03T18:44:51.430174+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:13:02.658508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:44:51.431464+0000","last_change":"2024-05-03T18:44:49.427176+0000","last_active":"2024-05-03T18:44:51.431464+0000","last_peered":"2024-05-03T18:44:51.431464+0000","last_clean":"2024-05-03T18:44:51.431464+0000","last_became_active":"2024-05-03T18:44:49.426546+0000","last_became_peered":"2024-05-03T18:44:49.426546+0000","last_unstale":"2024-05-03T18:44:51.431464+0000","last_undegraded":"2024-05-03T18:44:51.431464+0000","last_fullsized":"2024-05-03T18:44:51.431464+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:29:22.343376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:44:51.432297+0000","last_change":"2024-05-03T18:44:49.435037+0000","last_active":"2024-05-03T18:44:51.432297+0000","last_peered":"2024-05-03T18:44:51.432297+0000","last_clean":"2024-05-03T18:44:51.432297+0000","last_became_active":"2024-05-03T18:44:49.434693+0000","last_became_peered":"2024-05-03T18:44:49.434693+0000","last_unstale":"2024-05-03T18:44:51.432297+0000","last_undegraded":"2024-05-03T18:44:51.432297+0000","last_fullsized":"2024-05-03T18:44:51.432297+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:26:33.923608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"78'9","reported_seq":31,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:45:03.483324+0000","last_change":"2024-05-03T18:44:46.402606+0000","last_active":"2024-05-03T18:45:03.483324+0000","last_peered":"2024-05-03T18:45:03.483324+0000","last_clean":"2024-05-03T18:45:03.483324+0000","last_became_active":"2024-05-03T18:44:46.402245+0000","last_became_peered":"2024-05-03T18:44:46.402245+0000","last_unstale":"2024-05-03T18:45:03.483324+0000","last_undegraded":"2024-05-03T18:45:03.483324+0000","last_fullsized":"2024-05-03T18:45:03.483324+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_clean_scrub_stamp":"2024-05-03T18:44:45.380544+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:29:33.726119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:44:51.430853+0000","last_change":"2024-05-03T18:44:49.435279+0000","last_active":"2024-05-03T18:44:51.430853+0000","last_peered":"2024-05-03T18:44:51.430853+0000","last_clean":"2024-05-03T18:44:51.430853+0000","last_became_active":"2024-05-03T18:44:49.434779+0000","last_became_peered":"2024-05-03T18:44:49.434779+0000","last_unstale":"2024-05-03T18:44:51.430853+0000","last_undegraded":"2024-05-03T18:44:51.430853+0000","last_fullsized":"2024-05-03T18:44:51.430853+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:56:56.762051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"78'2","reported_seq":23,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:44:51.434991+0000","last_change":"2024-05-03T18:44:46.644899+0000","last_active":"2024-05-03T18:44:51.434991+0000","last_peered":"2024-05-03T18:44:51.434991+0000","last_clean":"2024-05-03T18:44:51.434991+0000","last_became_active":"2024-05-03T18:44:46.644508+0000","last_became_peered":"2024-05-03T18:44:46.644508+0000","last_unstale":"2024-05-03T18:44:51.434991+0000","last_undegraded":"2024-05-03T18:44:51.434991+0000","last_fullsized":"2024-05-03T18:44:51.434991+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_clean_scrub_stamp":"2024-05-03T18:44:45.380544+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T06:38:09.379634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:44:51.432839+0000","last_change":"2024-05-03T18:44:49.429720+0000","last_active":"2024-05-03T18:44:51.432839+0000","last_peered":"2024-05-03T18:44:51.432839+0000","last_clean":"2024-05-03T18:44:51.432839+0000","last_became_active":"2024-05-03T18:44:49.428399+0000","last_became_peered":"2024-05-03T18:44:49.428399+0000","last_unstale":"2024-05-03T18:44:51.432839+0000","last_undegraded":"2024-05-03T18:44:51.432839+0000","last_fullsized":"2024-05-03T18:44:51.432839+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:17:57.325926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"78'4","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:45:03.483080+0000","last_change":"2024-05-03T18:44:46.410397+0000","last_active":"2024-05-03T18:45:03.483080+0000","last_peered":"2024-05-03T18:45:03.483080+0000","last_clean":"2024-05-03T18:45:03.483080+0000","last_became_active":"2024-05-03T18:44:46.410030+0000","last_became_peered":"2024-05-03T18:44:46.410030+0000","last_unstale":"2024-05-03T18:45:03.483080+0000","last_undegraded":"2024-05-03T18:45:03.483080+0000","last_fullsized":"2024-05-03T18:45:03.483080+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_clean_scrub_stamp":"2024-05-03T18:44:45.380544+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:15:26.130070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:44:51.433958+0000","last_change":"2024-05-03T18:44:49.430335+0000","last_active":"2024-05-03T18:44:51.433958+0000","last_peered":"2024-05-03T18:44:51.433958+0000","last_clean":"2024-05-03T18:44:51.433958+0000","last_became_active":"2024-05-03T18:44:49.429991+0000","last_became_peered":"2024-05-03T18:44:49.429991+0000","last_unstale":"2024-05-03T18:44:51.433958+0000","last_undegraded":"2024-05-03T18:44:51.433958+0000","last_fullsized":"2024-05-03T18:44:51.433958+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:06:28.718138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"78'2","reported_seq":23,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:44:51.438736+0000","last_change":"2024-05-03T18:44:46.406552+0000","last_active":"2024-05-03T18:44:51.438736+0000","last_peered":"2024-05-03T18:44:51.438736+0000","last_clean":"2024-05-03T18:44:51.438736+0000","last_became_active":"2024-05-03T18:44:46.406160+0000","last_became_peered":"2024-05-03T18:44:46.406160+0000","last_unstale":"2024-05-03T18:44:51.438736+0000","last_undegraded":"2024-05-03T18:44:51.438736+0000","last_fullsized":"2024-05-03T18:44:51.438736+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_clean_scrub_stamp":"2024-05-03T18:44:45.380544+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:53:13.769225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:44:51.423057+0000","last_change":"2024-05-03T18:44:49.441301+0000","last_active":"2024-05-03T18:44:51.423057+0000","last_peered":"2024-05-03T18:44:51.423057+0000","last_clean":"2024-05-03T18:44:51.423057+0000","last_became_active":"2024-05-03T18:44:49.440936+0000","last_became_peered":"2024-05-03T18:44:49.440936+0000","last_unstale":"2024-05-03T18:44:51.423057+0000","last_undegraded":"2024-05-03T18:44:51.423057+0000","last_fullsized":"2024-05-03T18:44:51.423057+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:55:41.257325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"78'6","reported_seq":56,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:45:29.529151+0000","last_change":"2024-05-03T18:44:46.404827+0000","last_active":"2024-05-03T18:45:29.529151+0000","last_peered":"2024-05-03T18:45:29.529151+0000","last_clean":"2024-05-03T18:45:29.529151+0000","last_became_active":"2024-05-03T18:44:46.404455+0000","last_became_peered":"2024-05-03T18:44:46.404455+0000","last_unstale":"2024-05-03T18:45:29.529151+0000","last_undegraded":"2024-05-03T18:45:29.529151+0000","last_fullsized":"2024-05-03T18:45:29.529151+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_clean_scrub_stamp":"2024-05-03T18:44:45.380544+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:23:32.855160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":29,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:44:51.434308+0000","last_change":"2024-05-03T18:44:49.433656+0000","last_active":"2024-05-03T18:44:51.434308+0000","last_peered":"2024-05-03T18:44:51.434308+0000","last_clean":"2024-05-03T18:44:51.434308+0000","last_became_active":"2024-05-03T18:44:49.433326+0000","last_became_peered":"2024-05-03T18:44:49.433326+0000","last_unstale":"2024-05-03T18:44:51.434308+0000","last_undegraded":"2024-05-03T18:44:51.434308+0000","last_fullsized":"2024-05-03T18:44:51.434308+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:44:56.937334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"78'3","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:45:03.481450+0000","last_change":"2024-05-03T18:44:46.403960+0000","last_active":"2024-05-03T18:45:03.481450+0000","last_peered":"2024-05-03T18:45:03.481450+0000","last_clean":"2024-05-03T18:45:03.481450+0000","last_became_active":"2024-05-03T18:44:46.403560+0000","last_became_peered":"2024-05-03T18:44:46.403560+0000","last_unstale":"2024-05-03T18:45:03.481450+0000","last_undegraded":"2024-05-03T18:45:03.481450+0000","last_fullsized":"2024-05-03T18:45:03.481450+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_clean_scrub_stamp":"2024-05-03T18:44:45.380544+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:58:32.823640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:44:51.423234+0000","last_change":"2024-05-03T18:44:49.438621+0000","last_active":"2024-05-03T18:44:51.423234+0000","last_peered":"2024-05-03T18:44:51.423234+0000","last_clean":"2024-05-03T18:44:51.423234+0000","last_became_active":"2024-05-03T18:44:49.438288+0000","last_became_peered":"2024-05-03T18:44:49.438288+0000","last_unstale":"2024-05-03T18:44:51.423234+0000","last_undegraded":"2024-05-03T18:44:51.423234+0000","last_fullsized":"2024-05-03T18:44:51.423234+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:15:16.421982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"78'2","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:45:03.480995+0000","last_change":"2024-05-03T18:44:46.400282+0000","last_active":"2024-05-03T18:45:03.480995+0000","last_peered":"2024-05-03T18:45:03.480995+0000","last_clean":"2024-05-03T18:45:03.480995+0000","last_became_active":"2024-05-03T18:44:46.399873+0000","last_became_peered":"2024-05-03T18:44:46.399873+0000","last_unstale":"2024-05-03T18:45:03.480995+0000","last_undegraded":"2024-05-03T18:45:03.480995+0000","last_fullsized":"2024-05-03T18:45:03.480995+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_clean_scrub_stamp":"2024-05-03T18:44:45.380544+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:12:10.005145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:44:51.430037+0000","last_change":"2024-05-03T18:44:49.422311+0000","last_active":"2024-05-03T18:44:51.430037+0000","last_peered":"2024-05-03T18:44:51.430037+0000","last_clean":"2024-05-03T18:44:51.430037+0000","last_became_active":"2024-05-03T18:44:49.421962+0000","last_became_peered":"2024-05-03T18:44:49.421962+0000","last_unstale":"2024-05-03T18:44:51.430037+0000","last_undegraded":"2024-05-03T18:44:51.430037+0000","last_fullsized":"2024-05-03T18:44:51.430037+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:50:46.407989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"78'1","reported_seq":22,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:44:51.422507+0000","last_change":"2024-05-03T18:44:46.410969+0000","last_active":"2024-05-03T18:44:51.422507+0000","last_peered":"2024-05-03T18:44:51.422507+0000","last_clean":"2024-05-03T18:44:51.422507+0000","last_became_active":"2024-05-03T18:44:46.410089+0000","last_became_peered":"2024-05-03T18:44:46.410089+0000","last_unstale":"2024-05-03T18:44:51.422507+0000","last_undegraded":"2024-05-03T18:44:51.422507+0000","last_fullsized":"2024-05-03T18:44:51.422507+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_clean_scrub_stamp":"2024-05-03T18:44:45.380544+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:13:51.641235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:44:51.431355+0000","last_change":"2024-05-03T18:44:49.426946+0000","last_active":"2024-05-03T18:44:51.431355+0000","last_peered":"2024-05-03T18:44:51.431355+0000","last_clean":"2024-05-03T18:44:51.431355+0000","last_became_active":"2024-05-03T18:44:49.426609+0000","last_became_peered":"2024-05-03T18:44:49.426609+0000","last_unstale":"2024-05-03T18:44:51.431355+0000","last_undegraded":"2024-05-03T18:44:51.431355+0000","last_fullsized":"2024-05-03T18:44:51.431355+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:34:59.944769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"78'3","reported_seq":51,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:45:30.529939+0000","last_change":"2024-05-03T18:44:46.405415+0000","last_active":"2024-05-03T18:45:30.529939+0000","last_peered":"2024-05-03T18:45:30.529939+0000","last_clean":"2024-05-03T18:45:30.529939+0000","last_became_active":"2024-05-03T18:44:46.404928+0000","last_became_peered":"2024-05-03T18:44:46.404928+0000","last_unstale":"2024-05-03T18:45:30.529939+0000","last_undegraded":"2024-05-03T18:45:30.529939+0000","last_fullsized":"2024-05-03T18:45:30.529939+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_clean_scrub_stamp":"2024-05-03T18:44:45.380544+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:33:10.979432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":27,"num_read_kb":27,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:44:51.434912+0000","last_change":"2024-05-03T18:44:49.427873+0000","last_active":"2024-05-03T18:44:51.434912+0000","last_peered":"2024-05-03T18:44:51.434912+0000","last_clean":"2024-05-03T18:44:51.434912+0000","last_became_active":"2024-05-03T18:44:49.426797+0000","last_became_peered":"2024-05-03T18:44:49.426797+0000","last_unstale":"2024-05-03T18:44:51.434912+0000","last_undegraded":"2024-05-03T18:44:51.434912+0000","last_fullsized":"2024-05-03T18:44:51.434912+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:07:28.989454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"78'2","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:45:02.477284+0000","last_change":"2024-05-03T18:44:46.407167+0000","last_active":"2024-05-03T18:45:02.477284+0000","last_peered":"2024-05-03T18:45:02.477284+0000","last_clean":"2024-05-03T18:45:02.477284+0000","last_became_active":"2024-05-03T18:44:46.406787+0000","last_became_peered":"2024-05-03T18:44:46.406787+0000","last_unstale":"2024-05-03T18:45:02.477284+0000","last_undegraded":"2024-05-03T18:45:02.477284+0000","last_fullsized":"2024-05-03T18:45:02.477284+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_clean_scrub_stamp":"2024-05-03T18:44:45.380544+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:33:12.854056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:44:51.432715+0000","last_change":"2024-05-03T18:44:49.436339+0000","last_active":"2024-05-03T18:44:51.432715+0000","last_peered":"2024-05-03T18:44:51.432715+0000","last_clean":"2024-05-03T18:44:51.432715+0000","last_became_active":"2024-05-03T18:44:49.436029+0000","last_became_peered":"2024-05-03T18:44:49.436029+0000","last_unstale":"2024-05-03T18:44:51.432715+0000","last_undegraded":"2024-05-03T18:44:51.432715+0000","last_fullsized":"2024-05-03T18:44:51.432715+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:53:47.543701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"78'4","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:45:02.476035+0000","last_change":"2024-05-03T18:44:46.647273+0000","last_active":"2024-05-03T18:45:02.476035+0000","last_peered":"2024-05-03T18:45:02.476035+0000","last_clean":"2024-05-03T18:45:02.476035+0000","last_became_active":"2024-05-03T18:44:46.646881+0000","last_became_peered":"2024-05-03T18:44:46.646881+0000","last_unstale":"2024-05-03T18:45:02.476035+0000","last_undegraded":"2024-05-03T18:45:02.476035+0000","last_fullsized":"2024-05-03T18:45:02.476035+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_clean_scrub_stamp":"2024-05-03T18:44:45.380544+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:03:47.988180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:44:51.434279+0000","last_change":"2024-05-03T18:44:49.436525+0000","last_active":"2024-05-03T18:44:51.434279+0000","last_peered":"2024-05-03T18:44:51.434279+0000","last_clean":"2024-05-03T18:44:51.434279+0000","last_became_active":"2024-05-03T18:44:49.436109+0000","last_became_peered":"2024-05-03T18:44:49.436109+0000","last_unstale":"2024-05-03T18:44:51.434279+0000","last_undegraded":"2024-05-03T18:44:51.434279+0000","last_fullsized":"2024-05-03T18:44:51.434279+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:08:42.239248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"78'27","reported_seq":110,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:45:31.530889+0000","last_change":"2024-05-03T18:44:46.407936+0000","last_active":"2024-05-03T18:45:31.530889+0000","last_peered":"2024-05-03T18:45:31.530889+0000","last_clean":"2024-05-03T18:45:31.530889+0000","last_became_active":"2024-05-03T18:44:46.407504+0000","last_became_peered":"2024-05-03T18:44:46.407504+0000","last_unstale":"2024-05-03T18:45:31.530889+0000","last_undegraded":"2024-05-03T18:45:31.530889+0000","last_fullsized":"2024-05-03T18:45:31.530889+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_clean_scrub_stamp":"2024-05-03T18:44:45.380544+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:26:27.213572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":51833,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":62,"num_read_kb":81,"num_write":28,"num_write_kb":60,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:44:51.435119+0000","last_change":"2024-05-03T18:44:49.438509+0000","last_active":"2024-05-03T18:44:51.435119+0000","last_peered":"2024-05-03T18:44:51.435119+0000","last_clean":"2024-05-03T18:44:51.435119+0000","last_became_active":"2024-05-03T18:44:49.438185+0000","last_became_peered":"2024-05-03T18:44:49.438185+0000","last_unstale":"2024-05-03T18:44:51.435119+0000","last_undegraded":"2024-05-03T18:44:51.435119+0000","last_fullsized":"2024-05-03T18:44:51.435119+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:57:28.072095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"78'6","reported_seq":29,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:45:03.483761+0000","last_change":"2024-05-03T18:44:46.407842+0000","last_active":"2024-05-03T18:45:03.483761+0000","last_peered":"2024-05-03T18:45:03.483761+0000","last_clean":"2024-05-03T18:45:03.483761+0000","last_became_active":"2024-05-03T18:44:46.407506+0000","last_became_peered":"2024-05-03T18:44:46.407506+0000","last_unstale":"2024-05-03T18:45:03.483761+0000","last_undegraded":"2024-05-03T18:45:03.483761+0000","last_fullsized":"2024-05-03T18:45:03.483761+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_clean_scrub_stamp":"2024-05-03T18:44:45.380544+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:51:33.101539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:44:51.421973+0000","last_change":"2024-05-03T18:44:49.433214+0000","last_active":"2024-05-03T18:44:51.421973+0000","last_peered":"2024-05-03T18:44:51.421973+0000","last_clean":"2024-05-03T18:44:51.421973+0000","last_became_active":"2024-05-03T18:44:49.432879+0000","last_became_peered":"2024-05-03T18:44:49.432879+0000","last_unstale":"2024-05-03T18:44:51.421973+0000","last_undegraded":"2024-05-03T18:44:51.421973+0000","last_fullsized":"2024-05-03T18:44:51.421973+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:35:17.920304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"78'3","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:45:02.475561+0000","last_change":"2024-05-03T18:44:46.405280+0000","last_active":"2024-05-03T18:45:02.475561+0000","last_peered":"2024-05-03T18:45:02.475561+0000","last_clean":"2024-05-03T18:45:02.475561+0000","last_became_active":"2024-05-03T18:44:46.404126+0000","last_became_peered":"2024-05-03T18:44:46.404126+0000","last_unstale":"2024-05-03T18:45:02.475561+0000","last_undegraded":"2024-05-03T18:45:02.475561+0000","last_fullsized":"2024-05-03T18:45:02.475561+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_clean_scrub_stamp":"2024-05-03T18:44:45.380544+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:14:52.146440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:44:51.430134+0000","last_change":"2024-05-03T18:44:49.430582+0000","last_active":"2024-05-03T18:44:51.430134+0000","last_peered":"2024-05-03T18:44:51.430134+0000","last_clean":"2024-05-03T18:44:51.430134+0000","last_became_active":"2024-05-03T18:44:49.430246+0000","last_became_peered":"2024-05-03T18:44:49.430246+0000","last_unstale":"2024-05-03T18:44:51.430134+0000","last_undegraded":"2024-05-03T18:44:51.430134+0000","last_fullsized":"2024-05-03T18:44:51.430134+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:17:13.504586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"78'3","reported_seq":54,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:45:31.531420+0000","last_change":"2024-05-03T18:44:46.404276+0000","last_active":"2024-05-03T18:45:31.531420+0000","last_peered":"2024-05-03T18:45:31.531420+0000","last_clean":"2024-05-03T18:45:31.531420+0000","last_became_active":"2024-05-03T18:44:46.403843+0000","last_became_peered":"2024-05-03T18:44:46.403843+0000","last_unstale":"2024-05-03T18:45:31.531420+0000","last_undegraded":"2024-05-03T18:45:31.531420+0000","last_fullsized":"2024-05-03T18:45:31.531420+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_clean_scrub_stamp":"2024-05-03T18:44:45.380544+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:05:49.483444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":31,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:44:51.429449+0000","last_change":"2024-05-03T18:44:49.427950+0000","last_active":"2024-05-03T18:44:51.429449+0000","last_peered":"2024-05-03T18:44:51.429449+0000","last_clean":"2024-05-03T18:44:51.429449+0000","last_became_active":"2024-05-03T18:44:49.427469+0000","last_became_peered":"2024-05-03T18:44:49.427469+0000","last_unstale":"2024-05-03T18:44:51.429449+0000","last_undegraded":"2024-05-03T18:44:51.429449+0000","last_fullsized":"2024-05-03T18:44:51.429449+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:33:35.690809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"78'15","reported_seq":37,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:45:03.484759+0000","last_change":"2024-05-03T18:44:46.645189+0000","last_active":"2024-05-03T18:45:03.484759+0000","last_peered":"2024-05-03T18:45:03.484759+0000","last_clean":"2024-05-03T18:45:03.484759+0000","last_became_active":"2024-05-03T18:44:46.644642+0000","last_became_peered":"2024-05-03T18:44:46.644642+0000","last_unstale":"2024-05-03T18:45:03.484759+0000","last_undegraded":"2024-05-03T18:45:03.484759+0000","last_fullsized":"2024-05-03T18:45:03.484759+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_clean_scrub_stamp":"2024-05-03T18:44:45.380544+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:16:49.003568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:44:51.422530+0000","last_change":"2024-05-03T18:44:49.438931+0000","last_active":"2024-05-03T18:44:51.422530+0000","last_peered":"2024-05-03T18:44:51.422530+0000","last_clean":"2024-05-03T18:44:51.422530+0000","last_became_active":"2024-05-03T18:44:49.438573+0000","last_became_peered":"2024-05-03T18:44:49.438573+0000","last_unstale":"2024-05-03T18:44:51.422530+0000","last_undegraded":"2024-05-03T18:44:51.422530+0000","last_fullsized":"2024-05-03T18:44:51.422530+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T06:09:31.042649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"78'2","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:45:05.477902+0000","last_change":"2024-05-03T18:44:46.406752+0000","last_active":"2024-05-03T18:45:05.477902+0000","last_peered":"2024-05-03T18:45:05.477902+0000","last_clean":"2024-05-03T18:45:05.477902+0000","last_became_active":"2024-05-03T18:44:46.406288+0000","last_became_peered":"2024-05-03T18:44:46.406288+0000","last_unstale":"2024-05-03T18:45:05.477902+0000","last_undegraded":"2024-05-03T18:45:05.477902+0000","last_fullsized":"2024-05-03T18:45:05.477902+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_clean_scrub_stamp":"2024-05-03T18:44:45.380544+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:20:33.947006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:44:51.429712+0000","last_change":"2024-05-03T18:44:49.431103+0000","last_active":"2024-05-03T18:44:51.429712+0000","last_peered":"2024-05-03T18:44:51.429712+0000","last_clean":"2024-05-03T18:44:51.429712+0000","last_became_active":"2024-05-03T18:44:49.430776+0000","last_became_peered":"2024-05-03T18:44:49.430776+0000","last_unstale":"2024-05-03T18:44:51.429712+0000","last_undegraded":"2024-05-03T18:44:51.429712+0000","last_fullsized":"2024-05-03T18:44:51.429712+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:40:26.357553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":21,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:44:51.431245+0000","last_change":"2024-05-03T18:44:46.647155+0000","last_active":"2024-05-03T18:44:51.431245+0000","last_peered":"2024-05-03T18:44:51.431245+0000","last_clean":"2024-05-03T18:44:51.431245+0000","last_became_active":"2024-05-03T18:44:46.646719+0000","last_became_peered":"2024-05-03T18:44:46.646719+0000","last_unstale":"2024-05-03T18:44:51.431245+0000","last_undegraded":"2024-05-03T18:44:51.431245+0000","last_fullsized":"2024-05-03T18:44:51.431245+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_clean_scrub_stamp":"2024-05-03T18:44:45.380544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T18:49:13.701498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"78'1","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:44:52.425446+0000","last_change":"2024-05-03T18:44:49.423776+0000","last_active":"2024-05-03T18:44:52.425446+0000","last_peered":"2024-05-03T18:44:52.425446+0000","last_clean":"2024-05-03T18:44:52.425446+0000","last_became_active":"2024-05-03T18:44:49.423414+0000","last_became_peered":"2024-05-03T18:44:49.423414+0000","last_unstale":"2024-05-03T18:44:52.425446+0000","last_undegraded":"2024-05-03T18:44:52.425446+0000","last_fullsized":"2024-05-03T18:44:52.425446+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:48:03.556270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"78'1","reported_seq":23,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:44:51.430304+0000","last_change":"2024-05-03T18:44:46.407725+0000","last_active":"2024-05-03T18:44:51.430304+0000","last_peered":"2024-05-03T18:44:51.430304+0000","last_clean":"2024-05-03T18:44:51.430304+0000","last_became_active":"2024-05-03T18:44:46.407317+0000","last_became_peered":"2024-05-03T18:44:46.407317+0000","last_unstale":"2024-05-03T18:44:51.430304+0000","last_undegraded":"2024-05-03T18:44:51.430304+0000","last_fullsized":"2024-05-03T18:44:51.430304+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_clean_scrub_stamp":"2024-05-03T18:44:45.380544+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:02:32.739918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:44:51.430818+0000","last_change":"2024-05-03T18:44:49.432439+0000","last_active":"2024-05-03T18:44:51.430818+0000","last_peered":"2024-05-03T18:44:51.430818+0000","last_clean":"2024-05-03T18:44:51.430818+0000","last_became_active":"2024-05-03T18:44:49.431792+0000","last_became_peered":"2024-05-03T18:44:49.431792+0000","last_unstale":"2024-05-03T18:44:51.430818+0000","last_undegraded":"2024-05-03T18:44:51.430818+0000","last_fullsized":"2024-05-03T18:44:51.430818+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:12:46.223514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:44:51.422854+0000","last_change":"2024-05-03T18:44:49.440075+0000","last_active":"2024-05-03T18:44:51.422854+0000","last_peered":"2024-05-03T18:44:51.422854+0000","last_clean":"2024-05-03T18:44:51.422854+0000","last_became_active":"2024-05-03T18:44:49.439655+0000","last_became_peered":"2024-05-03T18:44:49.439655+0000","last_unstale":"2024-05-03T18:44:51.422854+0000","last_undegraded":"2024-05-03T18:44:51.422854+0000","last_fullsized":"2024-05-03T18:44:51.422854+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:19:03.306588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"78'4","reported_seq":51,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:45:28.526217+0000","last_change":"2024-05-03T18:44:46.645090+0000","last_active":"2024-05-03T18:45:28.526217+0000","last_peered":"2024-05-03T18:45:28.526217+0000","last_clean":"2024-05-03T18:45:28.526217+0000","last_became_active":"2024-05-03T18:44:46.644722+0000","last_became_peered":"2024-05-03T18:44:46.644722+0000","last_unstale":"2024-05-03T18:45:28.526217+0000","last_undegraded":"2024-05-03T18:45:28.526217+0000","last_fullsized":"2024-05-03T18:45:28.526217+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_clean_scrub_stamp":"2024-05-03T18:44:45.380544+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:44:38.032522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":25,"num_read_kb":25,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:44:51.430587+0000","last_change":"2024-05-03T18:44:49.432194+0000","last_active":"2024-05-03T18:44:51.430587+0000","last_peered":"2024-05-03T18:44:51.430587+0000","last_clean":"2024-05-03T18:44:51.430587+0000","last_became_active":"2024-05-03T18:44:49.431561+0000","last_became_peered":"2024-05-03T18:44:49.431561+0000","last_unstale":"2024-05-03T18:44:51.430587+0000","last_undegraded":"2024-05-03T18:44:51.430587+0000","last_fullsized":"2024-05-03T18:44:51.430587+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:30:41.506789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"78'3","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:45:05.490231+0000","last_change":"2024-05-03T18:44:46.402866+0000","last_active":"2024-05-03T18:45:05.490231+0000","last_peered":"2024-05-03T18:45:05.490231+0000","last_clean":"2024-05-03T18:45:05.490231+0000","last_became_active":"2024-05-03T18:44:46.402394+0000","last_became_peered":"2024-05-03T18:44:46.402394+0000","last_unstale":"2024-05-03T18:45:05.490231+0000","last_undegraded":"2024-05-03T18:45:05.490231+0000","last_fullsized":"2024-05-03T18:45:05.490231+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_clean_scrub_stamp":"2024-05-03T18:44:45.380544+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:28:37.257537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":96,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:44:51.429703+0000","last_change":"2024-05-03T18:43:28.584846+0000","last_active":"2024-05-03T18:44:51.429703+0000","last_peered":"2024-05-03T18:44:51.429703+0000","last_clean":"2024-05-03T18:44:51.429703+0000","last_became_active":"2024-05-03T18:43:28.580087+0000","last_became_peered":"2024-05-03T18:43:28.580087+0000","last_unstale":"2024-05-03T18:44:51.429703+0000","last_undegraded":"2024-05-03T18:44:51.429703+0000","last_fullsized":"2024-05-03T18:44:51.429703+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:40:08.599278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:40:08.599278+0000","last_clean_scrub_stamp":"2024-05-03T18:40:08.599278+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:14:31.836127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:44:51.421924+0000","last_change":"2024-05-03T18:44:49.438545+0000","last_active":"2024-05-03T18:44:51.421924+0000","last_peered":"2024-05-03T18:44:51.421924+0000","last_clean":"2024-05-03T18:44:51.421924+0000","last_became_active":"2024-05-03T18:44:49.438178+0000","last_became_peered":"2024-05-03T18:44:49.438178+0000","last_unstale":"2024-05-03T18:44:51.421924+0000","last_undegraded":"2024-05-03T18:44:51.421924+0000","last_fullsized":"2024-05-03T18:44:51.421924+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:55:10.320262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"78'3","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:45:03.482015+0000","last_change":"2024-05-03T18:44:46.409567+0000","last_active":"2024-05-03T18:45:03.482015+0000","last_peered":"2024-05-03T18:45:03.482015+0000","last_clean":"2024-05-03T18:45:03.482015+0000","last_became_active":"2024-05-03T18:44:46.409078+0000","last_became_peered":"2024-05-03T18:44:46.409078+0000","last_unstale":"2024-05-03T18:45:03.482015+0000","last_undegraded":"2024-05-03T18:45:03.482015+0000","last_fullsized":"2024-05-03T18:45:03.482015+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_clean_scrub_stamp":"2024-05-03T18:44:45.380544+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:51:45.865890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"78'1","reported_seq":23,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:45:02.475533+0000","last_change":"2024-05-03T18:44:46.645061+0000","last_active":"2024-05-03T18:45:02.475533+0000","last_peered":"2024-05-03T18:45:02.475533+0000","last_clean":"2024-05-03T18:45:02.475533+0000","last_became_active":"2024-05-03T18:44:46.644631+0000","last_became_peered":"2024-05-03T18:44:46.644631+0000","last_unstale":"2024-05-03T18:45:02.475533+0000","last_undegraded":"2024-05-03T18:45:02.475533+0000","last_fullsized":"2024-05-03T18:45:02.475533+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_clean_scrub_stamp":"2024-05-03T18:44:45.380544+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:19:37.285434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:44:51.422352+0000","last_change":"2024-05-03T18:44:49.435674+0000","last_active":"2024-05-03T18:44:51.422352+0000","last_peered":"2024-05-03T18:44:51.422352+0000","last_clean":"2024-05-03T18:44:51.422352+0000","last_became_active":"2024-05-03T18:44:49.435333+0000","last_became_peered":"2024-05-03T18:44:49.435333+0000","last_unstale":"2024-05-03T18:44:51.422352+0000","last_undegraded":"2024-05-03T18:44:51.422352+0000","last_fullsized":"2024-05-03T18:44:51.422352+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:05:20.283773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"78'10","reported_seq":31,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:45:23.431440+0000","last_change":"2024-05-03T18:44:46.402696+0000","last_active":"2024-05-03T18:45:23.431440+0000","last_peered":"2024-05-03T18:45:23.431440+0000","last_clean":"2024-05-03T18:45:23.431440+0000","last_became_active":"2024-05-03T18:44:46.402288+0000","last_became_peered":"2024-05-03T18:44:46.402288+0000","last_unstale":"2024-05-03T18:45:23.431440+0000","last_undegraded":"2024-05-03T18:45:23.431440+0000","last_fullsized":"2024-05-03T18:45:23.431440+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_clean_scrub_stamp":"2024-05-03T18:44:45.380544+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:45:32.200586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:44:51.433587+0000","last_change":"2024-05-03T18:44:49.422754+0000","last_active":"2024-05-03T18:44:51.433587+0000","last_peered":"2024-05-03T18:44:51.433587+0000","last_clean":"2024-05-03T18:44:51.433587+0000","last_became_active":"2024-05-03T18:44:49.422385+0000","last_became_peered":"2024-05-03T18:44:49.422385+0000","last_unstale":"2024-05-03T18:44:51.433587+0000","last_undegraded":"2024-05-03T18:44:51.433587+0000","last_fullsized":"2024-05-03T18:44:51.433587+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:07:35.065842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"78'4","reported_seq":25,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:45:02.474886+0000","last_change":"2024-05-03T18:44:47.135851+0000","last_active":"2024-05-03T18:45:02.474886+0000","last_peered":"2024-05-03T18:45:02.474886+0000","last_clean":"2024-05-03T18:45:02.474886+0000","last_became_active":"2024-05-03T18:44:47.135480+0000","last_became_peered":"2024-05-03T18:44:47.135480+0000","last_unstale":"2024-05-03T18:45:02.474886+0000","last_undegraded":"2024-05-03T18:45:02.474886+0000","last_fullsized":"2024-05-03T18:45:02.474886+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_clean_scrub_stamp":"2024-05-03T18:44:45.380544+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:50:55.582018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:44:51.432400+0000","last_change":"2024-05-03T18:44:49.433123+0000","last_active":"2024-05-03T18:44:51.432400+0000","last_peered":"2024-05-03T18:44:51.432400+0000","last_clean":"2024-05-03T18:44:51.432400+0000","last_became_active":"2024-05-03T18:44:49.432781+0000","last_became_peered":"2024-05-03T18:44:49.432781+0000","last_unstale":"2024-05-03T18:44:51.432400+0000","last_undegraded":"2024-05-03T18:44:51.432400+0000","last_fullsized":"2024-05-03T18:44:51.432400+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:30:49.781052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"78'2","reported_seq":23,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:45:02.477286+0000","last_change":"2024-05-03T18:44:46.645267+0000","last_active":"2024-05-03T18:45:02.477286+0000","last_peered":"2024-05-03T18:45:02.477286+0000","last_clean":"2024-05-03T18:45:02.477286+0000","last_became_active":"2024-05-03T18:44:46.644804+0000","last_became_peered":"2024-05-03T18:44:46.644804+0000","last_unstale":"2024-05-03T18:45:02.477286+0000","last_undegraded":"2024-05-03T18:45:02.477286+0000","last_fullsized":"2024-05-03T18:45:02.477286+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_clean_scrub_stamp":"2024-05-03T18:44:45.380544+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:10:59.262218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:44:51.432318+0000","last_change":"2024-05-03T18:44:49.439352+0000","last_active":"2024-05-03T18:44:51.432318+0000","last_peered":"2024-05-03T18:44:51.432318+0000","last_clean":"2024-05-03T18:44:51.432318+0000","last_became_active":"2024-05-03T18:44:49.438981+0000","last_became_peered":"2024-05-03T18:44:49.438981+0000","last_unstale":"2024-05-03T18:44:51.432318+0000","last_undegraded":"2024-05-03T18:44:51.432318+0000","last_fullsized":"2024-05-03T18:44:51.432318+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T06:04:25.566153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"78'3","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:45:02.476275+0000","last_change":"2024-05-03T18:44:46.402305+0000","last_active":"2024-05-03T18:45:02.476275+0000","last_peered":"2024-05-03T18:45:02.476275+0000","last_clean":"2024-05-03T18:45:02.476275+0000","last_became_active":"2024-05-03T18:44:46.401633+0000","last_became_peered":"2024-05-03T18:44:46.401633+0000","last_unstale":"2024-05-03T18:45:02.476275+0000","last_undegraded":"2024-05-03T18:45:02.476275+0000","last_fullsized":"2024-05-03T18:45:02.476275+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_clean_scrub_stamp":"2024-05-03T18:44:45.380544+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:55:06.025508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:44:51.429425+0000","last_change":"2024-05-03T18:44:49.436728+0000","last_active":"2024-05-03T18:44:51.429425+0000","last_peered":"2024-05-03T18:44:51.429425+0000","last_clean":"2024-05-03T18:44:51.429425+0000","last_became_active":"2024-05-03T18:44:49.436356+0000","last_became_peered":"2024-05-03T18:44:49.436356+0000","last_unstale":"2024-05-03T18:44:51.429425+0000","last_undegraded":"2024-05-03T18:44:51.429425+0000","last_fullsized":"2024-05-03T18:44:51.429425+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:31:34.827237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"78'1","reported_seq":23,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:44:51.433245+0000","last_change":"2024-05-03T18:44:46.404949+0000","last_active":"2024-05-03T18:44:51.433245+0000","last_peered":"2024-05-03T18:44:51.433245+0000","last_clean":"2024-05-03T18:44:51.433245+0000","last_became_active":"2024-05-03T18:44:46.404516+0000","last_became_peered":"2024-05-03T18:44:46.404516+0000","last_unstale":"2024-05-03T18:44:51.433245+0000","last_undegraded":"2024-05-03T18:44:51.433245+0000","last_fullsized":"2024-05-03T18:44:51.433245+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_clean_scrub_stamp":"2024-05-03T18:44:45.380544+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:02:16.070763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:44:51.430622+0000","last_change":"2024-05-03T18:44:49.424090+0000","last_active":"2024-05-03T18:44:51.430622+0000","last_peered":"2024-05-03T18:44:51.430622+0000","last_clean":"2024-05-03T18:44:51.430622+0000","last_became_active":"2024-05-03T18:44:49.423110+0000","last_became_peered":"2024-05-03T18:44:49.423110+0000","last_unstale":"2024-05-03T18:44:51.430622+0000","last_undegraded":"2024-05-03T18:44:51.430622+0000","last_fullsized":"2024-05-03T18:44:51.430622+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:27:19.600924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"78'4","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:45:02.478333+0000","last_change":"2024-05-03T18:44:46.410438+0000","last_active":"2024-05-03T18:45:02.478333+0000","last_peered":"2024-05-03T18:45:02.478333+0000","last_clean":"2024-05-03T18:45:02.478333+0000","last_became_active":"2024-05-03T18:44:46.410061+0000","last_became_peered":"2024-05-03T18:44:46.410061+0000","last_unstale":"2024-05-03T18:45:02.478333+0000","last_undegraded":"2024-05-03T18:45:02.478333+0000","last_fullsized":"2024-05-03T18:45:02.478333+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_clean_scrub_stamp":"2024-05-03T18:44:45.380544+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:30:57.668522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:44:51.430303+0000","last_change":"2024-05-03T18:44:49.432498+0000","last_active":"2024-05-03T18:44:51.430303+0000","last_peered":"2024-05-03T18:44:51.430303+0000","last_clean":"2024-05-03T18:44:51.430303+0000","last_became_active":"2024-05-03T18:44:49.432152+0000","last_became_peered":"2024-05-03T18:44:49.432152+0000","last_unstale":"2024-05-03T18:44:51.430303+0000","last_undegraded":"2024-05-03T18:44:51.430303+0000","last_fullsized":"2024-05-03T18:44:51.430303+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:07:35.058213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"78'3","reported_seq":26,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:45:03.483725+0000","last_change":"2024-05-03T18:44:47.135973+0000","last_active":"2024-05-03T18:45:03.483725+0000","last_peered":"2024-05-03T18:45:03.483725+0000","last_clean":"2024-05-03T18:45:03.483725+0000","last_became_active":"2024-05-03T18:44:47.135548+0000","last_became_peered":"2024-05-03T18:44:47.135548+0000","last_unstale":"2024-05-03T18:45:03.483725+0000","last_undegraded":"2024-05-03T18:45:03.483725+0000","last_fullsized":"2024-05-03T18:45:03.483725+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_clean_scrub_stamp":"2024-05-03T18:44:45.380544+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:06:42.858462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:44:51.430484+0000","last_change":"2024-05-03T18:44:49.432722+0000","last_active":"2024-05-03T18:44:51.430484+0000","last_peered":"2024-05-03T18:44:51.430484+0000","last_clean":"2024-05-03T18:44:51.430484+0000","last_became_active":"2024-05-03T18:44:49.432393+0000","last_became_peered":"2024-05-03T18:44:49.432393+0000","last_unstale":"2024-05-03T18:44:51.430484+0000","last_undegraded":"2024-05-03T18:44:51.430484+0000","last_fullsized":"2024-05-03T18:44:51.430484+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:44:20.551442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"78'1","reported_seq":23,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:44:51.433380+0000","last_change":"2024-05-03T18:44:46.402489+0000","last_active":"2024-05-03T18:44:51.433380+0000","last_peered":"2024-05-03T18:44:51.433380+0000","last_clean":"2024-05-03T18:44:51.433380+0000","last_became_active":"2024-05-03T18:44:46.401992+0000","last_became_peered":"2024-05-03T18:44:46.401992+0000","last_unstale":"2024-05-03T18:44:51.433380+0000","last_undegraded":"2024-05-03T18:44:51.433380+0000","last_fullsized":"2024-05-03T18:44:51.433380+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_clean_scrub_stamp":"2024-05-03T18:44:45.380544+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:09:10.600254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"78'1","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:44:51.430233+0000","last_change":"2024-05-03T18:44:49.430095+0000","last_active":"2024-05-03T18:44:51.430233+0000","last_peered":"2024-05-03T18:44:51.430233+0000","last_clean":"2024-05-03T18:44:51.430233+0000","last_became_active":"2024-05-03T18:44:49.429590+0000","last_became_peered":"2024-05-03T18:44:49.429590+0000","last_unstale":"2024-05-03T18:44:51.430233+0000","last_undegraded":"2024-05-03T18:44:51.430233+0000","last_fullsized":"2024-05-03T18:44:51.430233+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:50:29.362826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"78'2","reported_seq":24,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:44:51.430328+0000","last_change":"2024-05-03T18:44:46.404414+0000","last_active":"2024-05-03T18:44:51.430328+0000","last_peered":"2024-05-03T18:44:51.430328+0000","last_clean":"2024-05-03T18:44:51.430328+0000","last_became_active":"2024-05-03T18:44:46.404136+0000","last_became_peered":"2024-05-03T18:44:46.404136+0000","last_unstale":"2024-05-03T18:44:51.430328+0000","last_undegraded":"2024-05-03T18:44:51.430328+0000","last_fullsized":"2024-05-03T18:44:51.430328+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_clean_scrub_stamp":"2024-05-03T18:44:45.380544+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:24:58.125984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:44:51.430452+0000","last_change":"2024-05-03T18:44:49.429013+0000","last_active":"2024-05-03T18:44:51.430452+0000","last_peered":"2024-05-03T18:44:51.430452+0000","last_clean":"2024-05-03T18:44:51.430452+0000","last_became_active":"2024-05-03T18:44:49.428380+0000","last_became_peered":"2024-05-03T18:44:49.428380+0000","last_unstale":"2024-05-03T18:44:51.430452+0000","last_undegraded":"2024-05-03T18:44:51.430452+0000","last_fullsized":"2024-05-03T18:44:51.430452+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:13:00.073384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:44:51.429455+0000","last_change":"2024-05-03T18:44:49.437186+0000","last_active":"2024-05-03T18:44:51.429455+0000","last_peered":"2024-05-03T18:44:51.429455+0000","last_clean":"2024-05-03T18:44:51.429455+0000","last_became_active":"2024-05-03T18:44:49.435192+0000","last_became_peered":"2024-05-03T18:44:49.435192+0000","last_unstale":"2024-05-03T18:44:51.429455+0000","last_undegraded":"2024-05-03T18:44:51.429455+0000","last_fullsized":"2024-05-03T18:44:51.429455+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:16:25.707746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:44:51.431909+0000","last_change":"2024-05-03T18:44:49.441553+0000","last_active":"2024-05-03T18:44:51.431909+0000","last_peered":"2024-05-03T18:44:51.431909+0000","last_clean":"2024-05-03T18:44:51.431909+0000","last_became_active":"2024-05-03T18:44:49.441155+0000","last_became_peered":"2024-05-03T18:44:49.441155+0000","last_unstale":"2024-05-03T18:44:51.431909+0000","last_undegraded":"2024-05-03T18:44:51.431909+0000","last_fullsized":"2024-05-03T18:44:51.431909+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:53:53.888748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:44:51.431926+0000","last_change":"2024-05-03T18:44:49.436641+0000","last_active":"2024-05-03T18:44:51.431926+0000","last_peered":"2024-05-03T18:44:51.431926+0000","last_clean":"2024-05-03T18:44:51.431926+0000","last_became_active":"2024-05-03T18:44:49.436232+0000","last_became_peered":"2024-05-03T18:44:49.436232+0000","last_unstale":"2024-05-03T18:44:51.431926+0000","last_undegraded":"2024-05-03T18:44:51.431926+0000","last_fullsized":"2024-05-03T18:44:51.431926+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:06:55.773088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:44:51.429683+0000","last_change":"2024-05-03T18:44:49.434394+0000","last_active":"2024-05-03T18:44:51.429683+0000","last_peered":"2024-05-03T18:44:51.429683+0000","last_clean":"2024-05-03T18:44:51.429683+0000","last_became_active":"2024-05-03T18:44:49.433995+0000","last_became_peered":"2024-05-03T18:44:49.433995+0000","last_unstale":"2024-05-03T18:44:51.429683+0000","last_undegraded":"2024-05-03T18:44:51.429683+0000","last_fullsized":"2024-05-03T18:44:51.429683+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:25:20.478426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:44:51.422456+0000","last_change":"2024-05-03T18:44:49.439218+0000","last_active":"2024-05-03T18:44:51.422456+0000","last_peered":"2024-05-03T18:44:51.422456+0000","last_clean":"2024-05-03T18:44:51.422456+0000","last_became_active":"2024-05-03T18:44:49.438875+0000","last_became_peered":"2024-05-03T18:44:49.438875+0000","last_unstale":"2024-05-03T18:44:51.422456+0000","last_undegraded":"2024-05-03T18:44:51.422456+0000","last_fullsized":"2024-05-03T18:44:51.422456+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:08:55.899829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:44:51.429841+0000","last_change":"2024-05-03T18:44:49.433079+0000","last_active":"2024-05-03T18:44:51.429841+0000","last_peered":"2024-05-03T18:44:51.429841+0000","last_clean":"2024-05-03T18:44:51.429841+0000","last_became_active":"2024-05-03T18:44:49.432740+0000","last_became_peered":"2024-05-03T18:44:49.432740+0000","last_unstale":"2024-05-03T18:44:51.429841+0000","last_undegraded":"2024-05-03T18:44:51.429841+0000","last_fullsized":"2024-05-03T18:44:51.429841+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:31:36.317667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:44:51.431918+0000","last_change":"2024-05-03T18:44:49.435321+0000","last_active":"2024-05-03T18:44:51.431918+0000","last_peered":"2024-05-03T18:44:51.431918+0000","last_clean":"2024-05-03T18:44:51.431918+0000","last_became_active":"2024-05-03T18:44:49.434935+0000","last_became_peered":"2024-05-03T18:44:49.434935+0000","last_unstale":"2024-05-03T18:44:51.431918+0000","last_undegraded":"2024-05-03T18:44:51.431918+0000","last_fullsized":"2024-05-03T18:44:51.431918+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:14:32.406164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:44:51.430935+0000","last_change":"2024-05-03T18:44:49.438202+0000","last_active":"2024-05-03T18:44:51.430935+0000","last_peered":"2024-05-03T18:44:51.430935+0000","last_clean":"2024-05-03T18:44:51.430935+0000","last_became_active":"2024-05-03T18:44:49.437822+0000","last_became_peered":"2024-05-03T18:44:49.437822+0000","last_unstale":"2024-05-03T18:44:51.430935+0000","last_undegraded":"2024-05-03T18:44:51.430935+0000","last_fullsized":"2024-05-03T18:44:51.430935+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:12:54.417022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:44:51.422317+0000","last_change":"2024-05-03T18:44:49.434992+0000","last_active":"2024-05-03T18:44:51.422317+0000","last_peered":"2024-05-03T18:44:51.422317+0000","last_clean":"2024-05-03T18:44:51.422317+0000","last_became_active":"2024-05-03T18:44:49.434668+0000","last_became_peered":"2024-05-03T18:44:49.434668+0000","last_unstale":"2024-05-03T18:44:51.422317+0000","last_undegraded":"2024-05-03T18:44:51.422317+0000","last_fullsized":"2024-05-03T18:44:51.422317+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:20:48.527470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:44:51.430027+0000","last_change":"2024-05-03T18:44:49.432690+0000","last_active":"2024-05-03T18:44:51.430027+0000","last_peered":"2024-05-03T18:44:51.430027+0000","last_clean":"2024-05-03T18:44:51.430027+0000","last_became_active":"2024-05-03T18:44:49.432389+0000","last_became_peered":"2024-05-03T18:44:49.432389+0000","last_unstale":"2024-05-03T18:44:51.430027+0000","last_undegraded":"2024-05-03T18:44:51.430027+0000","last_fullsized":"2024-05-03T18:44:51.430027+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:51:27.326901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:44:51.433768+0000","last_change":"2024-05-03T18:44:49.434205+0000","last_active":"2024-05-03T18:44:51.433768+0000","last_peered":"2024-05-03T18:44:51.433768+0000","last_clean":"2024-05-03T18:44:51.433768+0000","last_became_active":"2024-05-03T18:44:49.433619+0000","last_became_peered":"2024-05-03T18:44:49.433619+0000","last_unstale":"2024-05-03T18:44:51.433768+0000","last_undegraded":"2024-05-03T18:44:51.433768+0000","last_fullsized":"2024-05-03T18:44:51.433768+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:36:15.933276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:44:51.422266+0000","last_change":"2024-05-03T18:44:49.425276+0000","last_active":"2024-05-03T18:44:51.422266+0000","last_peered":"2024-05-03T18:44:51.422266+0000","last_clean":"2024-05-03T18:44:51.422266+0000","last_became_active":"2024-05-03T18:44:49.424916+0000","last_became_peered":"2024-05-03T18:44:49.424916+0000","last_unstale":"2024-05-03T18:44:51.422266+0000","last_undegraded":"2024-05-03T18:44:51.422266+0000","last_fullsized":"2024-05-03T18:44:51.422266+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:09:03.512860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-05-03T18:44:51.429434+0000","last_change":"2024-05-03T18:44:49.433335+0000","last_active":"2024-05-03T18:44:51.429434+0000","last_peered":"2024-05-03T18:44:51.429434+0000","last_clean":"2024-05-03T18:44:51.429434+0000","last_became_active":"2024-05-03T18:44:49.432991+0000","last_became_peered":"2024-05-03T18:44:49.432991+0000","last_unstale":"2024-05-03T18:44:51.429434+0000","last_undegraded":"2024-05-03T18:44:51.429434+0000","last_fullsized":"2024-05-03T18:44:51.429434+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:31:25.182590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":55834,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":187,"num_read_kb":152,"num_write":159,"num_write_kb":101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":167502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":136,"ondisk_log_size":136,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710742,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27836,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717316,"statfs":{"total":95995035648,"available":95966531584,"internally_reserved":0,"allocated":839680,"data_stored":408028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906972,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28508,"kb_used_data":1492,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716644,"statfs":{"total":95995035648,"available":95965843456,"internally_reserved":0,"allocated":1527808,"data_stored":1051342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070496,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27852,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717300,"statfs":{"total":95995035648,"available":95966515200,"internally_reserved":0,"allocated":856064,"data_stored":408264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266725,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27852,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717300,"statfs":{"total":95995035648,"available":95966515200,"internally_reserved":0,"allocated":856064,"data_stored":408184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462955,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36752,"kb_used_data":1416,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708400,"statfs":{"total":95995035648,"available":95957401600,"internally_reserved":0,"allocated":1449984,"data_stored":1000114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626478,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27840,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717312,"statfs":{"total":95995035648,"available":95966527488,"internally_reserved":0,"allocated":843776,"data_stored":407653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574920,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27892,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717260,"statfs":{"total":95995035648,"available":95966474240,"internally_reserved":0,"allocated":897024,"data_stored":459279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738444,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36180,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708972,"statfs":{"total":95995035648,"available":95957987328,"internally_reserved":0,"allocated":864256,"data_stored":408924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411396,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":921600,"data_stored":461020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182463,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36176,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708976,"statfs":{"total":95995035648,"available":95957991424,"internally_reserved":0,"allocated":860160,"data_stored":407964,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986234,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27844,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717308,"statfs":{"total":95995035648,"available":95966523392,"internally_reserved":0,"allocated":847872,"data_stored":408090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822708,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36764,"kb_used_data":1428,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708388,"statfs":{"total":95995035648,"available":95957389312,"internally_reserved":0,"allocated":1462272,"data_stored":999798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":51833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":53574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":53411,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-03T18:45:34.683 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-05-03T18:45:34.683 INFO:tasks.cephadm:Setting up client nodes... 2024-05-03T18:45:34.683 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f500455e-097b-11ef-bc97-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-05-03T18:45:34.931 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:45:35.680 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:45:35 smithi032 ceph-mon[29763]: pgmap v302: 97 pgs: 97 active+clean; 631 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 2.9 KiB/s wr, 8 op/s 2024-05-03T18:45:35.680 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:45:35 smithi032 ceph-mon[29763]: from='client.14754 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-03T18:45:35.997 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:45:35 smithi045 ceph-mon[33299]: pgmap v302: 97 pgs: 97 active+clean; 631 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 2.9 KiB/s wr, 8 op/s 2024-05-03T18:45:35.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:45:35 smithi045 ceph-mon[33299]: from='client.14754 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-03T18:45:36.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:45:35 smithi113 ceph-mon[33540]: pgmap v302: 97 pgs: 97 active+clean; 631 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 2.9 KiB/s wr, 8 op/s 2024-05-03T18:45:36.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:45:35 smithi113 ceph-mon[33540]: from='client.14754 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-03T18:45:36.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:36.486+0000 7f1e52e1a700 1 -- 172.21.15.32:0/3030813391 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e4c103620 msgr2=0x7f1e4c103a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:36.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:36.486+0000 7f1e52e1a700 1 --2- 172.21.15.32:0/3030813391 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e4c103620 0x7f1e4c103a80 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f1e380099c0 tx=0x7f1e38009cd0 comp rx=0 tx=0).stop 2024-05-03T18:45:36.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:36.486+0000 7f1e52e1a700 1 -- 172.21.15.32:0/3030813391 shutdown_connections 2024-05-03T18:45:36.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:36.486+0000 7f1e52e1a700 1 --2- 172.21.15.32:0/3030813391 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1e4c103fc0 0x7f1e4c10a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:36.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:36.486+0000 7f1e52e1a700 1 --2- 172.21.15.32:0/3030813391 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1e4c102420 0x7f1e4c102840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:36.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:36.486+0000 7f1e52e1a700 1 --2- 172.21.15.32:0/3030813391 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e4c103620 0x7f1e4c103a80 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:36.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:36.486+0000 7f1e52e1a700 1 -- 172.21.15.32:0/3030813391 >> 172.21.15.32:0/3030813391 conn(0x7f1e4c0fd9c0 msgr2=0x7f1e4c0ffe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:45:36.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:36.486+0000 7f1e52e1a700 1 -- 172.21.15.32:0/3030813391 shutdown_connections 2024-05-03T18:45:36.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:36.487+0000 7f1e52e1a700 1 -- 172.21.15.32:0/3030813391 wait complete. 2024-05-03T18:45:36.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:36.487+0000 7f1e52e1a700 1 Processor -- start 2024-05-03T18:45:36.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:36.487+0000 7f1e52e1a700 1 -- start start 2024-05-03T18:45:36.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:36.487+0000 7f1e52e1a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e4c102420 0x7f1e4c06bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:36.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:36.487+0000 7f1e52e1a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1e4c103620 0x7f1e4c19e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:36.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:36.487+0000 7f1e52e1a700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1e4c103fc0 0x7f1e4c1a0900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:36.490 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:36.487+0000 7f1e52e1a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f1e4c10dea0 con 0x7f1e4c102420 2024-05-03T18:45:36.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:36.487+0000 7f1e52e1a700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f1e4c10dd20 con 0x7f1e4c103620 2024-05-03T18:45:36.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:36.487+0000 7f1e52e1a700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f1e4c10e020 con 0x7f1e4c103fc0 2024-05-03T18:45:36.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:36.487+0000 7f1e50bb6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e4c102420 0x7f1e4c06bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:36.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:36.487+0000 7f1e50bb6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e4c102420 0x7f1e4c06bce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38570/0 (socket says 172.21.15.32:38570) 2024-05-03T18:45:36.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:36.487+0000 7f1e50bb6700 1 -- 172.21.15.32:0/2518441608 learned_addr learned my addr 172.21.15.32:0/2518441608 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:45:36.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:36.488+0000 7f1e4bfff700 1 --2- 172.21.15.32:0/2518441608 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1e4c103620 0x7f1e4c19e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:36.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:36.488+0000 7f1e513b7700 1 --2- 172.21.15.32:0/2518441608 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1e4c103fc0 0x7f1e4c1a0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:36.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:36.488+0000 7f1e50bb6700 1 -- 172.21.15.32:0/2518441608 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1e4c103fc0 msgr2=0x7f1e4c1a0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:36.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:36.488+0000 7f1e50bb6700 1 --2- 172.21.15.32:0/2518441608 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1e4c103fc0 0x7f1e4c1a0900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:36.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:36.488+0000 7f1e50bb6700 1 -- 172.21.15.32:0/2518441608 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1e4c103620 msgr2=0x7f1e4c19e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:36.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:36.488+0000 7f1e50bb6700 1 --2- 172.21.15.32:0/2518441608 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1e4c103620 0x7f1e4c19e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:36.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:36.488+0000 7f1e50bb6700 1 -- 172.21.15.32:0/2518441608 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e4c1a11d0 con 0x7f1e4c102420 2024-05-03T18:45:36.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:36.488+0000 7f1e50bb6700 1 --2- 172.21.15.32:0/2518441608 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e4c102420 0x7f1e4c06bce0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f1e4000c990 tx=0x7f1e4000cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:45:36.491 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:36.488+0000 7f1e49ffb700 1 -- 172.21.15.32:0/2518441608 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e40007940 con 0x7f1e4c102420 2024-05-03T18:45:36.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:36.488+0000 7f1e49ffb700 1 -- 172.21.15.32:0/2518441608 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e4000f450 con 0x7f1e4c102420 2024-05-03T18:45:36.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:36.488+0000 7f1e52e1a700 1 -- 172.21.15.32:0/2518441608 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e4c1a14c0 con 0x7f1e4c102420 2024-05-03T18:45:36.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:36.488+0000 7f1e49ffb700 1 -- 172.21.15.32:0/2518441608 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e40018660 con 0x7f1e4c102420 2024-05-03T18:45:36.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:36.488+0000 7f1e52e1a700 1 -- 172.21.15.32:0/2518441608 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e4c1a19d0 con 0x7f1e4c102420 2024-05-03T18:45:36.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:36.489+0000 7f1e52e1a700 1 -- 172.21.15.32:0/2518441608 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e4c0519e0 con 0x7f1e4c102420 2024-05-03T18:45:36.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:36.491+0000 7f1e49ffb700 1 -- 172.21.15.32:0/2518441608 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f1e40007ae0 con 0x7f1e4c102420 2024-05-03T18:45:36.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:36.491+0000 7f1e49ffb700 1 --2- 172.21.15.32:0/2518441608 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f1e3c0b29b0 0x7f1e3c0b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:36.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:36.492+0000 7f1e4bfff700 1 --2- 172.21.15.32:0/2518441608 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f1e3c0b29b0 0x7f1e3c0b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:36.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:36.492+0000 7f1e49ffb700 1 -- 172.21.15.32:0/2518441608 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7979+0+0 (secure 0 0 0) 0x7f1e400e20e0 con 0x7f1e4c102420 2024-05-03T18:45:36.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:36.492+0000 7f1e4bfff700 1 --2- 172.21.15.32:0/2518441608 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f1e3c0b29b0 0x7f1e3c0b4e70 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f1e38009990 tx=0x7f1e3800b870 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:45:36.497 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:36.496+0000 7f1e49ffb700 1 -- 172.21.15.32:0/2518441608 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f1e40014030 con 0x7f1e4c102420 2024-05-03T18:45:36.800 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:36.798+0000 7f1e52e1a700 1 -- 172.21.15.32:0/2518441608 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f1e4c02e5b0 con 0x7f1e4c102420 2024-05-03T18:45:36.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:36.800+0000 7f1e49ffb700 1 -- 172.21.15.32:0/2518441608 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f1e4c02e5b0 con 0x7f1e4c102420 2024-05-03T18:45:36.806 INFO:teuthology.orchestra.run.smithi032.stdout:[client.0] 2024-05-03T18:45:36.806 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQDQMDVmDqixLxAA2rWTARgFM74vbCevQDY1nw== 2024-05-03T18:45:36.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:36.803+0000 7f1e52e1a700 1 -- 172.21.15.32:0/2518441608 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f1e3c0b29b0 msgr2=0x7f1e3c0b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:36.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:36.803+0000 7f1e52e1a700 1 --2- 172.21.15.32:0/2518441608 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f1e3c0b29b0 0x7f1e3c0b4e70 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f1e38009990 tx=0x7f1e3800b870 comp rx=0 tx=0).stop 2024-05-03T18:45:36.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:36.803+0000 7f1e52e1a700 1 -- 172.21.15.32:0/2518441608 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e4c102420 msgr2=0x7f1e4c06bce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:36.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:36.803+0000 7f1e52e1a700 1 --2- 172.21.15.32:0/2518441608 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e4c102420 0x7f1e4c06bce0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f1e4000c990 tx=0x7f1e4000cca0 comp rx=0 tx=0).stop 2024-05-03T18:45:36.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:36.803+0000 7f1e52e1a700 1 -- 172.21.15.32:0/2518441608 shutdown_connections 2024-05-03T18:45:36.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:36.803+0000 7f1e52e1a700 1 --2- 172.21.15.32:0/2518441608 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f1e3c0b29b0 0x7f1e3c0b4e70 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:36.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:36.803+0000 7f1e52e1a700 1 --2- 172.21.15.32:0/2518441608 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1e4c103fc0 0x7f1e4c1a0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:36.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:36.803+0000 7f1e52e1a700 1 --2- 172.21.15.32:0/2518441608 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e4c102420 0x7f1e4c06bce0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:36.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:36.803+0000 7f1e52e1a700 1 --2- 172.21.15.32:0/2518441608 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1e4c103620 0x7f1e4c19e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:36.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:36.803+0000 7f1e52e1a700 1 -- 172.21.15.32:0/2518441608 >> 172.21.15.32:0/2518441608 conn(0x7f1e4c0fd9c0 msgr2=0x7f1e4c106850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:45:36.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:36.803+0000 7f1e52e1a700 1 -- 172.21.15.32:0/2518441608 shutdown_connections 2024-05-03T18:45:36.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:36.803+0000 7f1e52e1a700 1 -- 172.21.15.32:0/2518441608 wait complete. 2024-05-03T18:45:37.421 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-05-03T18:45:37.421 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-05-03T18:45:37.421 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-05-03T18:45:37.461 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-05-03T18:45:37.461 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-05-03T18:45:37.462 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid f500455e-097b-11ef-bc97-c7b262605968 -- ceph mgr dump --format=json 2024-05-03T18:45:37.750 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:45:37.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:45:37 smithi032 ceph-mon[29763]: pgmap v303: 97 pgs: 97 active+clean; 631 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-05-03T18:45:37.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:45:37 smithi032 ceph-mon[29763]: from='client.? 172.21.15.32:0/2518441608' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-03T18:45:37.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:45:37 smithi032 ceph-mon[29763]: from='client.? 172.21.15.32:0/2518441608' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-03T18:45:37.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:45:37 smithi045 ceph-mon[33299]: pgmap v303: 97 pgs: 97 active+clean; 631 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-05-03T18:45:37.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:45:37 smithi045 ceph-mon[33299]: from='client.? 172.21.15.32:0/2518441608' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-03T18:45:37.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:45:37 smithi045 ceph-mon[33299]: from='client.? 172.21.15.32:0/2518441608' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-03T18:45:38.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:45:37 smithi113 ceph-mon[33540]: pgmap v303: 97 pgs: 97 active+clean; 631 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-05-03T18:45:38.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:45:37 smithi113 ceph-mon[33540]: from='client.? 172.21.15.32:0/2518441608' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-03T18:45:38.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:45:37 smithi113 ceph-mon[33540]: from='client.? 172.21.15.32:0/2518441608' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-03T18:45:39.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:39.260+0000 7fa9b7382700 1 -- 172.21.15.32:0/3210824569 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9b0102b40 msgr2=0x7fa9b0102fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:39.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:39.260+0000 7fa9b7382700 1 --2- 172.21.15.32:0/3210824569 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9b0102b40 0x7fa9b0102fc0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fa9a0009990 tx=0x7fa9a0009ca0 comp rx=0 tx=0).stop 2024-05-03T18:45:39.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:39.260+0000 7fa9b7382700 1 -- 172.21.15.32:0/3210824569 shutdown_connections 2024-05-03T18:45:39.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:39.260+0000 7fa9b7382700 1 --2- 172.21.15.32:0/3210824569 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa9b0103500 0x7fa9b010f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:39.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:39.260+0000 7fa9b7382700 1 --2- 172.21.15.32:0/3210824569 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa9b0107390 0x7fa9b0107770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:39.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:39.260+0000 7fa9b7382700 1 --2- 172.21.15.32:0/3210824569 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9b0102b40 0x7fa9b0102fc0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:39.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:39.260+0000 7fa9b7382700 1 -- 172.21.15.32:0/3210824569 >> 172.21.15.32:0/3210824569 conn(0x7fa9b00789a0 msgr2=0x7fa9b00ffd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:45:39.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:39.260+0000 7fa9b7382700 1 -- 172.21.15.32:0/3210824569 shutdown_connections 2024-05-03T18:45:39.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:39.260+0000 7fa9b7382700 1 -- 172.21.15.32:0/3210824569 wait complete. 2024-05-03T18:45:39.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:39.261+0000 7fa9b7382700 1 Processor -- start 2024-05-03T18:45:39.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:39.261+0000 7fa9b7382700 1 -- start start 2024-05-03T18:45:39.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:39.261+0000 7fa9b7382700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9b0102b40 0x7fa9b019f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:39.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:39.261+0000 7fa9b7382700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa9b0103500 0x7fa9b019fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:39.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:39.261+0000 7fa9b7382700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa9b0107390 0x7fa9b0199a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:39.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:39.261+0000 7fa9b7382700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa9b01126a0 con 0x7fa9b0102b40 2024-05-03T18:45:39.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:39.261+0000 7fa9b7382700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa9b0112520 con 0x7fa9b0107390 2024-05-03T18:45:39.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:39.261+0000 7fa9b7382700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fa9b0112820 con 0x7fa9b0103500 2024-05-03T18:45:39.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:39.261+0000 7fa9b511e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9b0102b40 0x7fa9b019f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:39.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:39.261+0000 7fa9b511e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9b0102b40 0x7fa9b019f860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38590/0 (socket says 172.21.15.32:38590) 2024-05-03T18:45:39.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:39.261+0000 7fa9b511e700 1 -- 172.21.15.32:0/2250653205 learned_addr learned my addr 172.21.15.32:0/2250653205 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:45:39.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:39.261+0000 7fa9b591f700 1 --2- 172.21.15.32:0/2250653205 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa9b0107390 0x7fa9b0199a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:39.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:39.261+0000 7fa9b491d700 1 --2- 172.21.15.32:0/2250653205 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa9b0103500 0x7fa9b019fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:39.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:39.261+0000 7fa9b511e700 1 -- 172.21.15.32:0/2250653205 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa9b0103500 msgr2=0x7fa9b019fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:39.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:39.261+0000 7fa9b511e700 1 --2- 172.21.15.32:0/2250653205 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa9b0103500 0x7fa9b019fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:39.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:39.261+0000 7fa9b511e700 1 -- 172.21.15.32:0/2250653205 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa9b0107390 msgr2=0x7fa9b0199a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:39.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:39.261+0000 7fa9b511e700 1 --2- 172.21.15.32:0/2250653205 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa9b0107390 0x7fa9b0199a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:39.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:39.261+0000 7fa9b511e700 1 -- 172.21.15.32:0/2250653205 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9b019a180 con 0x7fa9b0102b40 2024-05-03T18:45:39.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:39.262+0000 7fa9b511e700 1 --2- 172.21.15.32:0/2250653205 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9b0102b40 0x7fa9b019f860 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac00f9e0 tx=0x7fa9ac00fcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:45:39.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:39.262+0000 7fa9a67fc700 1 -- 172.21.15.32:0/2250653205 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9ac0116d0 con 0x7fa9b0102b40 2024-05-03T18:45:39.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:39.262+0000 7fa9a67fc700 1 -- 172.21.15.32:0/2250653205 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9ac011d20 con 0x7fa9b0102b40 2024-05-03T18:45:39.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:39.262+0000 7fa9b7382700 1 -- 172.21.15.32:0/2250653205 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9b019a470 con 0x7fa9b0102b40 2024-05-03T18:45:39.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:39.262+0000 7fa9a67fc700 1 -- 172.21.15.32:0/2250653205 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9ac01adb0 con 0x7fa9b0102b40 2024-05-03T18:45:39.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:39.262+0000 7fa9b7382700 1 -- 172.21.15.32:0/2250653205 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9b00fdd10 con 0x7fa9b0102b40 2024-05-03T18:45:39.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:39.263+0000 7fa9b7382700 1 -- 172.21.15.32:0/2250653205 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9b006a190 con 0x7fa9b0102b40 2024-05-03T18:45:39.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:39.264+0000 7fa9a67fc700 1 -- 172.21.15.32:0/2250653205 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fa9ac011870 con 0x7fa9b0102b40 2024-05-03T18:45:39.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:39.265+0000 7fa9a67fc700 1 --2- 172.21.15.32:0/2250653205 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fa99c0b2990 0x7fa99c0b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:39.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:39.265+0000 7fa9b491d700 1 --2- 172.21.15.32:0/2250653205 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fa99c0b2990 0x7fa99c0b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:39.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:39.265+0000 7fa9a67fc700 1 -- 172.21.15.32:0/2250653205 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7979+0+0 (secure 0 0 0) 0x7fa9ac0e5ff0 con 0x7fa9b0102b40 2024-05-03T18:45:39.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:39.265+0000 7fa9b491d700 1 --2- 172.21.15.32:0/2250653205 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fa99c0b2990 0x7fa99c0b4e50 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa9a0005cd0 tx=0x7fa9a0005c40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:45:39.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:39.268+0000 7fa9a67fc700 1 -- 172.21.15.32:0/2250653205 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fa9ac0adc60 con 0x7fa9b0102b40 2024-05-03T18:45:39.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:39.552+0000 7fa9b7382700 1 -- 172.21.15.32:0/2250653205 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7fa9b00fdfd0 con 0x7fa9b0102b40 2024-05-03T18:45:39.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:39.558+0000 7fa9a67fc700 1 -- 172.21.15.32:0/2250653205 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+288646 (secure 0 0 0) 0x7fa9ac0ad560 con 0x7fa9b0102b40 2024-05-03T18:45:39.562 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-03T18:45:39.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:39.561+0000 7fa9b7382700 1 -- 172.21.15.32:0/2250653205 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fa99c0b2990 msgr2=0x7fa99c0b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:39.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:39.561+0000 7fa9b7382700 1 --2- 172.21.15.32:0/2250653205 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fa99c0b2990 0x7fa99c0b4e50 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa9a0005cd0 tx=0x7fa9a0005c40 comp rx=0 tx=0).stop 2024-05-03T18:45:39.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:39.561+0000 7fa9b7382700 1 -- 172.21.15.32:0/2250653205 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9b0102b40 msgr2=0x7fa9b019f860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:39.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:39.561+0000 7fa9b7382700 1 --2- 172.21.15.32:0/2250653205 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9b0102b40 0x7fa9b019f860 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac00f9e0 tx=0x7fa9ac00fcf0 comp rx=0 tx=0).stop 2024-05-03T18:45:39.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:39.561+0000 7fa9b7382700 1 -- 172.21.15.32:0/2250653205 shutdown_connections 2024-05-03T18:45:39.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:39.561+0000 7fa9b7382700 1 --2- 172.21.15.32:0/2250653205 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fa99c0b2990 0x7fa99c0b4e50 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:39.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:39.561+0000 7fa9b7382700 1 --2- 172.21.15.32:0/2250653205 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa9b0107390 0x7fa9b0199a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:39.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:39.561+0000 7fa9b7382700 1 --2- 172.21.15.32:0/2250653205 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9b0102b40 0x7fa9b019f860 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:39.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:39.561+0000 7fa9b7382700 1 --2- 172.21.15.32:0/2250653205 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa9b0103500 0x7fa9b019fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:39.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:39.561+0000 7fa9b7382700 1 -- 172.21.15.32:0/2250653205 >> 172.21.15.32:0/2250653205 conn(0x7fa9b00789a0 msgr2=0x7fa9b0101c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:45:39.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:39.561+0000 7fa9b7382700 1 -- 172.21.15.32:0/2250653205 shutdown_connections 2024-05-03T18:45:39.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:39.561+0000 7fa9b7382700 1 -- 172.21.15.32:0/2250653205 wait complete. 2024-05-03T18:45:39.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:45:39 smithi032 ceph-mon[29763]: pgmap v304: 97 pgs: 97 active+clean; 631 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-05-03T18:45:39.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:45:39 smithi032 ceph-mon[29763]: from='client.? 172.21.15.32:0/2250653205' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-03T18:45:39.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:45:39 smithi045 ceph-mon[33299]: pgmap v304: 97 pgs: 97 active+clean; 631 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-05-03T18:45:39.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:45:39 smithi045 ceph-mon[33299]: from='client.? 172.21.15.32:0/2250653205' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-03T18:45:40.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:45:39 smithi113 ceph-mon[33540]: pgmap v304: 97 pgs: 97 active+clean; 631 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-05-03T18:45:40.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:45:39 smithi113 ceph-mon[33540]: from='client.? 172.21.15.32:0/2250653205' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-03T18:45:40.167 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":1796547904},{"type":"v1","addr":"172.21.15.32:6801","nonce":1796547904}]},"active_addr":"172.21.15.32:6801/1796547904","active_change":"2024-05-03T18:37:45.879715+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24116,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.32:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.32:0","nonce":2420643319}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.32:0","nonce":4241040506}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.32:0","nonce":1612163949}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.32:0","nonce":2548560223}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.32:0","nonce":274334939}]}]} 2024-05-03T18:45:40.173 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-05-03T18:45:40.173 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-05-03T18:45:40.173 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid f500455e-097b-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-03T18:45:40.423 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:45:41.815 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:45:41 smithi032 ceph-mon[29763]: pgmap v305: 97 pgs: 97 active+clean; 631 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-03T18:45:41.997 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:45:41 smithi045 ceph-mon[33299]: pgmap v305: 97 pgs: 97 active+clean; 631 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-03T18:45:42.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:45:41 smithi113 ceph-mon[33540]: pgmap v305: 97 pgs: 97 active+clean; 631 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-03T18:45:42.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:42.135+0000 7fbf86da0700 1 -- 172.21.15.32:0/838530414 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf80103750 msgr2=0x7fbf80103b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:42.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:42.135+0000 7fbf86da0700 1 --2- 172.21.15.32:0/838530414 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf80103750 0x7fbf80103b30 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fbf740099c0 tx=0x7fbf74009cd0 comp rx=0 tx=0).stop 2024-05-03T18:45:42.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:42.135+0000 7fbf86da0700 1 -- 172.21.15.32:0/838530414 shutdown_connections 2024-05-03T18:45:42.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:42.135+0000 7fbf86da0700 1 --2- 172.21.15.32:0/838530414 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf80108850 0x7fbf801103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:42.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:42.135+0000 7fbf86da0700 1 --2- 172.21.15.32:0/838530414 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf80103750 0x7fbf80103b30 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:42.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:42.135+0000 7fbf86da0700 1 --2- 172.21.15.32:0/838530414 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbf80104100 0x7fbf80108120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:42.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:42.135+0000 7fbf86da0700 1 -- 172.21.15.32:0/838530414 >> 172.21.15.32:0/838530414 conn(0x7fbf8007a600 msgr2=0x7fbf8007aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:45:42.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:42.135+0000 7fbf86da0700 1 -- 172.21.15.32:0/838530414 shutdown_connections 2024-05-03T18:45:42.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:42.135+0000 7fbf86da0700 1 -- 172.21.15.32:0/838530414 wait complete. 2024-05-03T18:45:42.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:42.136+0000 7fbf86da0700 1 Processor -- start 2024-05-03T18:45:42.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:42.136+0000 7fbf86da0700 1 -- start start 2024-05-03T18:45:42.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:42.136+0000 7fbf86da0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf80103750 0x7fbf8019f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:42.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:42.136+0000 7fbf86da0700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbf80104100 0x7fbf8019fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:42.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:42.136+0000 7fbf86da0700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf80108850 0x7fbf801999c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:42.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:42.136+0000 7fbf86da0700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fbf80113640 con 0x7fbf80103750 2024-05-03T18:45:42.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:42.136+0000 7fbf86da0700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fbf801134c0 con 0x7fbf80108850 2024-05-03T18:45:42.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:42.136+0000 7fbf86da0700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fbf801137c0 con 0x7fbf80104100 2024-05-03T18:45:42.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:42.136+0000 7fbf84b3c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf80103750 0x7fbf8019f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:42.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:42.136+0000 7fbf84b3c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf80103750 0x7fbf8019f7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38600/0 (socket says 172.21.15.32:38600) 2024-05-03T18:45:42.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:42.136+0000 7fbf84b3c700 1 -- 172.21.15.32:0/133852862 learned_addr learned my addr 172.21.15.32:0/133852862 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:45:42.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:42.136+0000 7fbf7ffff700 1 --2- 172.21.15.32:0/133852862 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbf80104100 0x7fbf8019fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:42.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:42.137+0000 7fbf8533d700 1 --2- 172.21.15.32:0/133852862 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf80108850 0x7fbf801999c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:42.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:42.137+0000 7fbf84b3c700 1 -- 172.21.15.32:0/133852862 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbf80104100 msgr2=0x7fbf8019fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:42.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:42.137+0000 7fbf84b3c700 1 --2- 172.21.15.32:0/133852862 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbf80104100 0x7fbf8019fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:42.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:42.137+0000 7fbf84b3c700 1 -- 172.21.15.32:0/133852862 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf80108850 msgr2=0x7fbf801999c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:42.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:42.137+0000 7fbf84b3c700 1 --2- 172.21.15.32:0/133852862 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf80108850 0x7fbf801999c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:42.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:42.137+0000 7fbf84b3c700 1 -- 172.21.15.32:0/133852862 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf8019a120 con 0x7fbf80103750 2024-05-03T18:45:42.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:42.137+0000 7fbf8533d700 1 --2- 172.21.15.32:0/133852862 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf80108850 0x7fbf801999c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:45:42.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:42.137+0000 7fbf84b3c700 1 --2- 172.21.15.32:0/133852862 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf80103750 0x7fbf8019f7b0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fbf7400b580 tx=0x7fbf740047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:45:42.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:42.137+0000 7fbf7dffb700 1 -- 172.21.15.32:0/133852862 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf74027070 con 0x7fbf80103750 2024-05-03T18:45:42.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:42.137+0000 7fbf86da0700 1 -- 172.21.15.32:0/133852862 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf8019a3b0 con 0x7fbf80103750 2024-05-03T18:45:42.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:42.137+0000 7fbf7dffb700 1 -- 172.21.15.32:0/133852862 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf74003f20 con 0x7fbf80103750 2024-05-03T18:45:42.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:42.137+0000 7fbf7dffb700 1 -- 172.21.15.32:0/133852862 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf74022d70 con 0x7fbf80103750 2024-05-03T18:45:42.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:42.137+0000 7fbf86da0700 1 -- 172.21.15.32:0/133852862 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf801a6680 con 0x7fbf80103750 2024-05-03T18:45:42.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:42.138+0000 7fbf86da0700 1 -- 172.21.15.32:0/133852862 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf80104be0 con 0x7fbf80103750 2024-05-03T18:45:42.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:42.140+0000 7fbf7dffb700 1 -- 172.21.15.32:0/133852862 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fbf74022500 con 0x7fbf80103750 2024-05-03T18:45:42.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:42.140+0000 7fbf7dffb700 1 --2- 172.21.15.32:0/133852862 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fbf700b2910 0x7fbf700b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:42.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:42.141+0000 7fbf7ffff700 1 --2- 172.21.15.32:0/133852862 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fbf700b2910 0x7fbf700b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:42.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:42.141+0000 7fbf7dffb700 1 -- 172.21.15.32:0/133852862 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7979+0+0 (secure 0 0 0) 0x7fbf740e32b0 con 0x7fbf80103750 2024-05-03T18:45:42.142 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:42.141+0000 7fbf7ffff700 1 --2- 172.21.15.32:0/133852862 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fbf700b2910 0x7fbf700b4dd0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fbf80103090 tx=0x7fbf6c00b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:45:42.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:42.144+0000 7fbf7dffb700 1 -- 172.21.15.32:0/133852862 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fbf74013070 con 0x7fbf80103750 2024-05-03T18:45:42.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:42.363+0000 7fbf86da0700 1 -- 172.21.15.32:0/133852862 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbf8002e510 con 0x7fbf80103750 2024-05-03T18:45:42.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:42.365+0000 7fbf7dffb700 1 -- 172.21.15.32:0/133852862 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) ==== 74+0+22148 (secure 0 0 0) 0x7fbf740ab0b0 con 0x7fbf80103750 2024-05-03T18:45:42.366 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-03T18:45:42.366 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":78,"fsid":"f500455e-097b-11ef-bc97-c7b262605968","created":"2024-05-03T18:36:43.750772+0000","modified":"2024-05-03T18:44:51.155086+0000","last_up_change":"2024-05-03T18:43:50.933818+0000","last_in_change":"2024-05-03T18:43:31.804386+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-03T18:40:07.981853+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-03T18:44:44.607622+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-03T18:44:48.077391+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0430c3c0-2c50-471c-9c74-2869a0627c74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":262324956},{"type":"v1","addr":"172.21.15.32:6803","nonce":262324956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":262324956},{"type":"v1","addr":"172.21.15.32:6805","nonce":262324956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":262324956},{"type":"v1","addr":"172.21.15.32:6809","nonce":262324956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":262324956},{"type":"v1","addr":"172.21.15.32:6807","nonce":262324956}]},"public_addr":"172.21.15.32:6803/262324956","cluster_addr":"172.21.15.32:6805/262324956","heartbeat_back_addr":"172.21.15.32:6809/262324956","heartbeat_front_addr":"172.21.15.32:6807/262324956","state":["exists","up"]},{"osd":1,"uuid":"ba8f0b67-1d7d-4d4c-8276-eff664d21f02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1209907302},{"type":"v1","addr":"172.21.15.45:6801","nonce":1209907302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1209907302},{"type":"v1","addr":"172.21.15.45:6803","nonce":1209907302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1209907302},{"type":"v1","addr":"172.21.15.45:6807","nonce":1209907302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1209907302},{"type":"v1","addr":"172.21.15.45:6805","nonce":1209907302}]},"public_addr":"172.21.15.45:6801/1209907302","cluster_addr":"172.21.15.45:6803/1209907302","heartbeat_back_addr":"172.21.15.45:6807/1209907302","heartbeat_front_addr":"172.21.15.45:6805/1209907302","state":["exists","up"]},{"osd":2,"uuid":"5624eabc-be47-4bb8-9d8e-b84734769c83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":2308128926},{"type":"v1","addr":"172.21.15.113:6801","nonce":2308128926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":2308128926},{"type":"v1","addr":"172.21.15.113:6803","nonce":2308128926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":2308128926},{"type":"v1","addr":"172.21.15.113:6807","nonce":2308128926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":2308128926},{"type":"v1","addr":"172.21.15.113:6805","nonce":2308128926}]},"public_addr":"172.21.15.113:6801/2308128926","cluster_addr":"172.21.15.113:6803/2308128926","heartbeat_back_addr":"172.21.15.113:6807/2308128926","heartbeat_front_addr":"172.21.15.113:6805/2308128926","state":["exists","up"]},{"osd":3,"uuid":"62c9c977-afe1-4112-beba-b25af7caa2f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":2606514063},{"type":"v1","addr":"172.21.15.32:6811","nonce":2606514063}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":2606514063},{"type":"v1","addr":"172.21.15.32:6813","nonce":2606514063}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":2606514063},{"type":"v1","addr":"172.21.15.32:6817","nonce":2606514063}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":2606514063},{"type":"v1","addr":"172.21.15.32:6815","nonce":2606514063}]},"public_addr":"172.21.15.32:6811/2606514063","cluster_addr":"172.21.15.32:6813/2606514063","heartbeat_back_addr":"172.21.15.32:6817/2606514063","heartbeat_front_addr":"172.21.15.32:6815/2606514063","state":["exists","up"]},{"osd":4,"uuid":"99ad449f-b0bc-4aa1-b0cf-4a7a458126e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":71932118},{"type":"v1","addr":"172.21.15.45:6809","nonce":71932118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":71932118},{"type":"v1","addr":"172.21.15.45:6811","nonce":71932118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":71932118},{"type":"v1","addr":"172.21.15.45:6815","nonce":71932118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":71932118},{"type":"v1","addr":"172.21.15.45:6813","nonce":71932118}]},"public_addr":"172.21.15.45:6809/71932118","cluster_addr":"172.21.15.45:6811/71932118","heartbeat_back_addr":"172.21.15.45:6815/71932118","heartbeat_front_addr":"172.21.15.45:6813/71932118","state":["exists","up"]},{"osd":5,"uuid":"a6688ade-47d7-4a52-807f-3c880a8ffcbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":4085018597},{"type":"v1","addr":"172.21.15.113:6809","nonce":4085018597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":4085018597},{"type":"v1","addr":"172.21.15.113:6811","nonce":4085018597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":4085018597},{"type":"v1","addr":"172.21.15.113:6815","nonce":4085018597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":4085018597},{"type":"v1","addr":"172.21.15.113:6813","nonce":4085018597}]},"public_addr":"172.21.15.113:6809/4085018597","cluster_addr":"172.21.15.113:6811/4085018597","heartbeat_back_addr":"172.21.15.113:6815/4085018597","heartbeat_front_addr":"172.21.15.113:6813/4085018597","state":["exists","up"]},{"osd":6,"uuid":"1d9ebda9-55c7-473a-9dc0-fc60845bbfe9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":791407796},{"type":"v1","addr":"172.21.15.32:6819","nonce":791407796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":791407796},{"type":"v1","addr":"172.21.15.32:6821","nonce":791407796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":791407796},{"type":"v1","addr":"172.21.15.32:6825","nonce":791407796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":791407796},{"type":"v1","addr":"172.21.15.32:6823","nonce":791407796}]},"public_addr":"172.21.15.32:6819/791407796","cluster_addr":"172.21.15.32:6821/791407796","heartbeat_back_addr":"172.21.15.32:6825/791407796","heartbeat_front_addr":"172.21.15.32:6823/791407796","state":["exists","up"]},{"osd":7,"uuid":"988cce65-23ed-4814-ac09-177eb2f89fe1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1032572438},{"type":"v1","addr":"172.21.15.45:6817","nonce":1032572438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1032572438},{"type":"v1","addr":"172.21.15.45:6819","nonce":1032572438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1032572438},{"type":"v1","addr":"172.21.15.45:6823","nonce":1032572438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1032572438},{"type":"v1","addr":"172.21.15.45:6821","nonce":1032572438}]},"public_addr":"172.21.15.45:6817/1032572438","cluster_addr":"172.21.15.45:6819/1032572438","heartbeat_back_addr":"172.21.15.45:6823/1032572438","heartbeat_front_addr":"172.21.15.45:6821/1032572438","state":["exists","up"]},{"osd":8,"uuid":"816e8ce8-5afa-4b52-89be-f7793ed12a0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3569181558},{"type":"v1","addr":"172.21.15.113:6817","nonce":3569181558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3569181558},{"type":"v1","addr":"172.21.15.113:6819","nonce":3569181558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3569181558},{"type":"v1","addr":"172.21.15.113:6823","nonce":3569181558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3569181558},{"type":"v1","addr":"172.21.15.113:6821","nonce":3569181558}]},"public_addr":"172.21.15.113:6817/3569181558","cluster_addr":"172.21.15.113:6819/3569181558","heartbeat_back_addr":"172.21.15.113:6823/3569181558","heartbeat_front_addr":"172.21.15.113:6821/3569181558","state":["exists","up"]},{"osd":9,"uuid":"1d35c435-36b8-40e8-a2d4-0efe904f1107","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":4290196345},{"type":"v1","addr":"172.21.15.32:6827","nonce":4290196345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":4290196345},{"type":"v1","addr":"172.21.15.32:6829","nonce":4290196345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":4290196345},{"type":"v1","addr":"172.21.15.32:6833","nonce":4290196345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":4290196345},{"type":"v1","addr":"172.21.15.32:6831","nonce":4290196345}]},"public_addr":"172.21.15.32:6827/4290196345","cluster_addr":"172.21.15.32:6829/4290196345","heartbeat_back_addr":"172.21.15.32:6833/4290196345","heartbeat_front_addr":"172.21.15.32:6831/4290196345","state":["exists","up"]},{"osd":10,"uuid":"9af7879e-aa8d-4ef8-b289-4b145b614820","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1817064361},{"type":"v1","addr":"172.21.15.45:6825","nonce":1817064361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1817064361},{"type":"v1","addr":"172.21.15.45:6827","nonce":1817064361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1817064361},{"type":"v1","addr":"172.21.15.45:6831","nonce":1817064361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1817064361},{"type":"v1","addr":"172.21.15.45:6829","nonce":1817064361}]},"public_addr":"172.21.15.45:6825/1817064361","cluster_addr":"172.21.15.45:6827/1817064361","heartbeat_back_addr":"172.21.15.45:6831/1817064361","heartbeat_front_addr":"172.21.15.45:6829/1817064361","state":["exists","up"]},{"osd":11,"uuid":"22981709-1f9d-46f6-892a-5678394a2e1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":703464306},{"type":"v1","addr":"172.21.15.113:6825","nonce":703464306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":703464306},{"type":"v1","addr":"172.21.15.113:6827","nonce":703464306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":703464306},{"type":"v1","addr":"172.21.15.113:6831","nonce":703464306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":703464306},{"type":"v1","addr":"172.21.15.113:6829","nonce":703464306}]},"public_addr":"172.21.15.113:6825/703464306","cluster_addr":"172.21.15.113:6827/703464306","heartbeat_back_addr":"172.21.15.113:6831/703464306","heartbeat_front_addr":"172.21.15.113:6829/703464306","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:39:18.455575+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:39:42.234466+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:40:05.843072+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:40:30.822533+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:40:54.952487+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:41:18.750184+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:41:43.403326+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:42:08.285730+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:42:32.861740+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:42:59.307277+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:43:24.509691+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:43:49.850077+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:0/743758274":"2024-05-04T18:37:45.879602+0000","172.21.15.32:6801/2019702098":"2024-05-04T18:37:45.879602+0000","172.21.15.32:6800/2019702098":"2024-05-04T18:37:45.879602+0000","172.21.15.32:0/3192385439":"2024-05-04T18:37:18.492319+0000","172.21.15.32:0/1450308494":"2024-05-04T18:37:18.492319+0000","172.21.15.32:0/3957699240":"2024-05-04T18:37:45.879602+0000","172.21.15.32:0/3310503393":"2024-05-04T18:37:18.492319+0000","172.21.15.32:6801/3945405684":"2024-05-04T18:37:18.492319+0000","172.21.15.32:6800/3945405684":"2024-05-04T18:37:18.492319+0000","172.21.15.32:0/2471164468":"2024-05-04T18:37:45.879602+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-03T18:45:42.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:42.367+0000 7fbf86da0700 1 -- 172.21.15.32:0/133852862 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fbf700b2910 msgr2=0x7fbf700b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:42.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:42.367+0000 7fbf86da0700 1 --2- 172.21.15.32:0/133852862 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fbf700b2910 0x7fbf700b4dd0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fbf80103090 tx=0x7fbf6c00b3c0 comp rx=0 tx=0).stop 2024-05-03T18:45:42.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:42.367+0000 7fbf86da0700 1 -- 172.21.15.32:0/133852862 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf80103750 msgr2=0x7fbf8019f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:42.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:42.367+0000 7fbf86da0700 1 --2- 172.21.15.32:0/133852862 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf80103750 0x7fbf8019f7b0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fbf7400b580 tx=0x7fbf740047b0 comp rx=0 tx=0).stop 2024-05-03T18:45:42.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:42.367+0000 7fbf86da0700 1 -- 172.21.15.32:0/133852862 shutdown_connections 2024-05-03T18:45:42.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:42.367+0000 7fbf86da0700 1 --2- 172.21.15.32:0/133852862 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fbf700b2910 0x7fbf700b4dd0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:42.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:42.367+0000 7fbf86da0700 1 --2- 172.21.15.32:0/133852862 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf80108850 0x7fbf801999c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:42.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:42.367+0000 7fbf86da0700 1 --2- 172.21.15.32:0/133852862 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf80103750 0x7fbf8019f7b0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:42.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:42.367+0000 7fbf86da0700 1 --2- 172.21.15.32:0/133852862 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbf80104100 0x7fbf8019fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:42.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:42.367+0000 7fbf86da0700 1 -- 172.21.15.32:0/133852862 >> 172.21.15.32:0/133852862 conn(0x7fbf8007a600 msgr2=0x7fbf801099a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:45:42.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:42.367+0000 7fbf86da0700 1 -- 172.21.15.32:0/133852862 shutdown_connections 2024-05-03T18:45:42.369 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:42.367+0000 7fbf86da0700 1 -- 172.21.15.32:0/133852862 wait complete. 2024-05-03T18:45:42.734 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:45:42 smithi032 ceph-mon[29763]: from='client.? 172.21.15.32:0/133852862' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T18:45:42.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:45:42 smithi045 ceph-mon[33299]: from='client.? 172.21.15.32:0/133852862' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T18:45:43.022 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-05-03T18:45:43.023 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid f500455e-097b-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-03T18:45:43.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:45:42 smithi113 ceph-mon[33540]: from='client.? 172.21.15.32:0/133852862' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T18:45:43.271 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:45:43.663 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:45:43 smithi032 ceph-mon[29763]: pgmap v306: 97 pgs: 97 active+clean; 631 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-03T18:45:43.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:45:43 smithi045 ceph-mon[33299]: pgmap v306: 97 pgs: 97 active+clean; 631 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-03T18:45:44.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:45:43 smithi113 ceph-mon[33540]: pgmap v306: 97 pgs: 97 active+clean; 631 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-03T18:45:44.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:44.774+0000 7f74a45c3700 1 -- 172.21.15.32:0/2562542856 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f749c104130 msgr2=0x7f749c108140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:44.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:44.774+0000 7f74a45c3700 1 --2- 172.21.15.32:0/2562542856 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f749c104130 0x7f749c108140 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f74980099c0 tx=0x7f7498009cd0 comp rx=0 tx=0).stop 2024-05-03T18:45:44.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:44.774+0000 7f74a45c3700 1 -- 172.21.15.32:0/2562542856 shutdown_connections 2024-05-03T18:45:44.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:44.774+0000 7f74a45c3700 1 --2- 172.21.15.32:0/2562542856 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f749c108870 0x7f749c1103f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:44.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:44.774+0000 7f74a45c3700 1 --2- 172.21.15.32:0/2562542856 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f749c103780 0x7f749c103b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:44.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:44.774+0000 7f74a45c3700 1 --2- 172.21.15.32:0/2562542856 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f749c104130 0x7f749c108140 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:44.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:44.774+0000 7f74a45c3700 1 -- 172.21.15.32:0/2562542856 >> 172.21.15.32:0/2562542856 conn(0x7f749c07a610 msgr2=0x7f749c07aa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:45:44.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:44.774+0000 7f74a45c3700 1 -- 172.21.15.32:0/2562542856 shutdown_connections 2024-05-03T18:45:44.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:44.774+0000 7f74a45c3700 1 -- 172.21.15.32:0/2562542856 wait complete. 2024-05-03T18:45:44.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:44.775+0000 7f74a45c3700 1 Processor -- start 2024-05-03T18:45:44.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:44.775+0000 7f74a45c3700 1 -- start start 2024-05-03T18:45:44.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:44.775+0000 7f74a45c3700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f749c103780 0x7f749c19f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:44.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:44.775+0000 7f74a45c3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f749c104130 0x7f749c19fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:44.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:44.775+0000 7f74a45c3700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f749c108870 0x7f749c199910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:44.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:44.775+0000 7f74a45c3700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f749c113650 con 0x7f749c104130 2024-05-03T18:45:44.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:44.775+0000 7f74a45c3700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f749c1134d0 con 0x7f749c103780 2024-05-03T18:45:44.778 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:44.775+0000 7f74a45c3700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f749c1137d0 con 0x7f749c108870 2024-05-03T18:45:44.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:44.775+0000 7f74a1b5e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f749c104130 0x7f749c19fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:44.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:44.775+0000 7f74a1b5e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f749c104130 0x7f749c19fc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44642/0 (socket says 172.21.15.32:44642) 2024-05-03T18:45:44.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:44.775+0000 7f74a1b5e700 1 -- 172.21.15.32:0/3894972247 learned_addr learned my addr 172.21.15.32:0/3894972247 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:45:44.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:44.776+0000 7f74a2b60700 1 --2- 172.21.15.32:0/3894972247 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f749c108870 0x7f749c199910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:44.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:44.776+0000 7f74a235f700 1 --2- 172.21.15.32:0/3894972247 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f749c103780 0x7f749c19f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:44.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:44.776+0000 7f74a1b5e700 1 -- 172.21.15.32:0/3894972247 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f749c108870 msgr2=0x7f749c199910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:44.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:44.776+0000 7f74a1b5e700 1 --2- 172.21.15.32:0/3894972247 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f749c108870 0x7f749c199910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:44.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:44.776+0000 7f74a1b5e700 1 -- 172.21.15.32:0/3894972247 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f749c103780 msgr2=0x7f749c19f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:44.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:44.776+0000 7f74a1b5e700 1 --2- 172.21.15.32:0/3894972247 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f749c103780 0x7f749c19f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:44.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:44.776+0000 7f74a1b5e700 1 -- 172.21.15.32:0/3894972247 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f749c19a070 con 0x7f749c104130 2024-05-03T18:45:44.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:44.776+0000 7f74a235f700 1 --2- 172.21.15.32:0/3894972247 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f749c103780 0x7f749c19f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:45:44.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:44.776+0000 7f74a1b5e700 1 --2- 172.21.15.32:0/3894972247 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f749c104130 0x7f749c19fc90 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f749800b580 tx=0x7f74980047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:45:44.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:44.776+0000 7f74937fe700 1 -- 172.21.15.32:0/3894972247 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7498027070 con 0x7f749c104130 2024-05-03T18:45:44.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:44.776+0000 7f74a45c3700 1 -- 172.21.15.32:0/3894972247 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f749c19a300 con 0x7f749c104130 2024-05-03T18:45:44.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:44.776+0000 7f74937fe700 1 -- 172.21.15.32:0/3894972247 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7498004020 con 0x7f749c104130 2024-05-03T18:45:44.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:44.776+0000 7f74937fe700 1 -- 172.21.15.32:0/3894972247 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7498022e30 con 0x7f749c104130 2024-05-03T18:45:44.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:44.776+0000 7f74a45c3700 1 -- 172.21.15.32:0/3894972247 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f749c1a6620 con 0x7f749c104130 2024-05-03T18:45:44.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:44.777+0000 7f74a45c3700 1 -- 172.21.15.32:0/3894972247 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f749c06a190 con 0x7f749c104130 2024-05-03T18:45:44.780 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:44.779+0000 7f74937fe700 1 -- 172.21.15.32:0/3894972247 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f7498022590 con 0x7f749c104130 2024-05-03T18:45:44.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:44.780+0000 7f74937fe700 1 --2- 172.21.15.32:0/3894972247 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f74880b2910 0x7f74880b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:44.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:44.780+0000 7f74a235f700 1 --2- 172.21.15.32:0/3894972247 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f74880b2910 0x7f74880b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:44.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:44.780+0000 7f74937fe700 1 -- 172.21.15.32:0/3894972247 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7979+0+0 (secure 0 0 0) 0x7f74980e33f0 con 0x7f749c104130 2024-05-03T18:45:44.781 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:44.780+0000 7f74a235f700 1 --2- 172.21.15.32:0/3894972247 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f74880b2910 0x7f74880b4dd0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f749c19aa50 tx=0x7f748c009670 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:45:44.784 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:44.783+0000 7f74937fe700 1 -- 172.21.15.32:0/3894972247 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f7498030070 con 0x7f749c104130 2024-05-03T18:45:45.010 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:45.009+0000 7f74a45c3700 1 -- 172.21.15.32:0/3894972247 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f749c02e510 con 0x7f749c104130 2024-05-03T18:45:45.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:45.011+0000 7f74937fe700 1 -- 172.21.15.32:0/3894972247 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) ==== 74+0+22148 (secure 0 0 0) 0x7f7498013070 con 0x7f749c104130 2024-05-03T18:45:45.012 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-03T18:45:45.012 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":78,"fsid":"f500455e-097b-11ef-bc97-c7b262605968","created":"2024-05-03T18:36:43.750772+0000","modified":"2024-05-03T18:44:51.155086+0000","last_up_change":"2024-05-03T18:43:50.933818+0000","last_in_change":"2024-05-03T18:43:31.804386+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-03T18:40:07.981853+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-03T18:44:44.607622+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-03T18:44:48.077391+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0430c3c0-2c50-471c-9c74-2869a0627c74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":262324956},{"type":"v1","addr":"172.21.15.32:6803","nonce":262324956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":262324956},{"type":"v1","addr":"172.21.15.32:6805","nonce":262324956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":262324956},{"type":"v1","addr":"172.21.15.32:6809","nonce":262324956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":262324956},{"type":"v1","addr":"172.21.15.32:6807","nonce":262324956}]},"public_addr":"172.21.15.32:6803/262324956","cluster_addr":"172.21.15.32:6805/262324956","heartbeat_back_addr":"172.21.15.32:6809/262324956","heartbeat_front_addr":"172.21.15.32:6807/262324956","state":["exists","up"]},{"osd":1,"uuid":"ba8f0b67-1d7d-4d4c-8276-eff664d21f02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1209907302},{"type":"v1","addr":"172.21.15.45:6801","nonce":1209907302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1209907302},{"type":"v1","addr":"172.21.15.45:6803","nonce":1209907302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1209907302},{"type":"v1","addr":"172.21.15.45:6807","nonce":1209907302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1209907302},{"type":"v1","addr":"172.21.15.45:6805","nonce":1209907302}]},"public_addr":"172.21.15.45:6801/1209907302","cluster_addr":"172.21.15.45:6803/1209907302","heartbeat_back_addr":"172.21.15.45:6807/1209907302","heartbeat_front_addr":"172.21.15.45:6805/1209907302","state":["exists","up"]},{"osd":2,"uuid":"5624eabc-be47-4bb8-9d8e-b84734769c83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":2308128926},{"type":"v1","addr":"172.21.15.113:6801","nonce":2308128926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":2308128926},{"type":"v1","addr":"172.21.15.113:6803","nonce":2308128926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":2308128926},{"type":"v1","addr":"172.21.15.113:6807","nonce":2308128926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":2308128926},{"type":"v1","addr":"172.21.15.113:6805","nonce":2308128926}]},"public_addr":"172.21.15.113:6801/2308128926","cluster_addr":"172.21.15.113:6803/2308128926","heartbeat_back_addr":"172.21.15.113:6807/2308128926","heartbeat_front_addr":"172.21.15.113:6805/2308128926","state":["exists","up"]},{"osd":3,"uuid":"62c9c977-afe1-4112-beba-b25af7caa2f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":2606514063},{"type":"v1","addr":"172.21.15.32:6811","nonce":2606514063}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":2606514063},{"type":"v1","addr":"172.21.15.32:6813","nonce":2606514063}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":2606514063},{"type":"v1","addr":"172.21.15.32:6817","nonce":2606514063}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":2606514063},{"type":"v1","addr":"172.21.15.32:6815","nonce":2606514063}]},"public_addr":"172.21.15.32:6811/2606514063","cluster_addr":"172.21.15.32:6813/2606514063","heartbeat_back_addr":"172.21.15.32:6817/2606514063","heartbeat_front_addr":"172.21.15.32:6815/2606514063","state":["exists","up"]},{"osd":4,"uuid":"99ad449f-b0bc-4aa1-b0cf-4a7a458126e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":71932118},{"type":"v1","addr":"172.21.15.45:6809","nonce":71932118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":71932118},{"type":"v1","addr":"172.21.15.45:6811","nonce":71932118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":71932118},{"type":"v1","addr":"172.21.15.45:6815","nonce":71932118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":71932118},{"type":"v1","addr":"172.21.15.45:6813","nonce":71932118}]},"public_addr":"172.21.15.45:6809/71932118","cluster_addr":"172.21.15.45:6811/71932118","heartbeat_back_addr":"172.21.15.45:6815/71932118","heartbeat_front_addr":"172.21.15.45:6813/71932118","state":["exists","up"]},{"osd":5,"uuid":"a6688ade-47d7-4a52-807f-3c880a8ffcbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":4085018597},{"type":"v1","addr":"172.21.15.113:6809","nonce":4085018597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":4085018597},{"type":"v1","addr":"172.21.15.113:6811","nonce":4085018597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":4085018597},{"type":"v1","addr":"172.21.15.113:6815","nonce":4085018597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":4085018597},{"type":"v1","addr":"172.21.15.113:6813","nonce":4085018597}]},"public_addr":"172.21.15.113:6809/4085018597","cluster_addr":"172.21.15.113:6811/4085018597","heartbeat_back_addr":"172.21.15.113:6815/4085018597","heartbeat_front_addr":"172.21.15.113:6813/4085018597","state":["exists","up"]},{"osd":6,"uuid":"1d9ebda9-55c7-473a-9dc0-fc60845bbfe9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":791407796},{"type":"v1","addr":"172.21.15.32:6819","nonce":791407796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":791407796},{"type":"v1","addr":"172.21.15.32:6821","nonce":791407796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":791407796},{"type":"v1","addr":"172.21.15.32:6825","nonce":791407796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":791407796},{"type":"v1","addr":"172.21.15.32:6823","nonce":791407796}]},"public_addr":"172.21.15.32:6819/791407796","cluster_addr":"172.21.15.32:6821/791407796","heartbeat_back_addr":"172.21.15.32:6825/791407796","heartbeat_front_addr":"172.21.15.32:6823/791407796","state":["exists","up"]},{"osd":7,"uuid":"988cce65-23ed-4814-ac09-177eb2f89fe1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1032572438},{"type":"v1","addr":"172.21.15.45:6817","nonce":1032572438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1032572438},{"type":"v1","addr":"172.21.15.45:6819","nonce":1032572438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1032572438},{"type":"v1","addr":"172.21.15.45:6823","nonce":1032572438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1032572438},{"type":"v1","addr":"172.21.15.45:6821","nonce":1032572438}]},"public_addr":"172.21.15.45:6817/1032572438","cluster_addr":"172.21.15.45:6819/1032572438","heartbeat_back_addr":"172.21.15.45:6823/1032572438","heartbeat_front_addr":"172.21.15.45:6821/1032572438","state":["exists","up"]},{"osd":8,"uuid":"816e8ce8-5afa-4b52-89be-f7793ed12a0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3569181558},{"type":"v1","addr":"172.21.15.113:6817","nonce":3569181558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3569181558},{"type":"v1","addr":"172.21.15.113:6819","nonce":3569181558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3569181558},{"type":"v1","addr":"172.21.15.113:6823","nonce":3569181558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3569181558},{"type":"v1","addr":"172.21.15.113:6821","nonce":3569181558}]},"public_addr":"172.21.15.113:6817/3569181558","cluster_addr":"172.21.15.113:6819/3569181558","heartbeat_back_addr":"172.21.15.113:6823/3569181558","heartbeat_front_addr":"172.21.15.113:6821/3569181558","state":["exists","up"]},{"osd":9,"uuid":"1d35c435-36b8-40e8-a2d4-0efe904f1107","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":4290196345},{"type":"v1","addr":"172.21.15.32:6827","nonce":4290196345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":4290196345},{"type":"v1","addr":"172.21.15.32:6829","nonce":4290196345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":4290196345},{"type":"v1","addr":"172.21.15.32:6833","nonce":4290196345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":4290196345},{"type":"v1","addr":"172.21.15.32:6831","nonce":4290196345}]},"public_addr":"172.21.15.32:6827/4290196345","cluster_addr":"172.21.15.32:6829/4290196345","heartbeat_back_addr":"172.21.15.32:6833/4290196345","heartbeat_front_addr":"172.21.15.32:6831/4290196345","state":["exists","up"]},{"osd":10,"uuid":"9af7879e-aa8d-4ef8-b289-4b145b614820","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1817064361},{"type":"v1","addr":"172.21.15.45:6825","nonce":1817064361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1817064361},{"type":"v1","addr":"172.21.15.45:6827","nonce":1817064361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1817064361},{"type":"v1","addr":"172.21.15.45:6831","nonce":1817064361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1817064361},{"type":"v1","addr":"172.21.15.45:6829","nonce":1817064361}]},"public_addr":"172.21.15.45:6825/1817064361","cluster_addr":"172.21.15.45:6827/1817064361","heartbeat_back_addr":"172.21.15.45:6831/1817064361","heartbeat_front_addr":"172.21.15.45:6829/1817064361","state":["exists","up"]},{"osd":11,"uuid":"22981709-1f9d-46f6-892a-5678394a2e1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":703464306},{"type":"v1","addr":"172.21.15.113:6825","nonce":703464306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":703464306},{"type":"v1","addr":"172.21.15.113:6827","nonce":703464306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":703464306},{"type":"v1","addr":"172.21.15.113:6831","nonce":703464306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":703464306},{"type":"v1","addr":"172.21.15.113:6829","nonce":703464306}]},"public_addr":"172.21.15.113:6825/703464306","cluster_addr":"172.21.15.113:6827/703464306","heartbeat_back_addr":"172.21.15.113:6831/703464306","heartbeat_front_addr":"172.21.15.113:6829/703464306","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:39:18.455575+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:39:42.234466+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:40:05.843072+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:40:30.822533+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:40:54.952487+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:41:18.750184+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:41:43.403326+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:42:08.285730+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:42:32.861740+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:42:59.307277+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:43:24.509691+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:43:49.850077+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:0/743758274":"2024-05-04T18:37:45.879602+0000","172.21.15.32:6801/2019702098":"2024-05-04T18:37:45.879602+0000","172.21.15.32:6800/2019702098":"2024-05-04T18:37:45.879602+0000","172.21.15.32:0/3192385439":"2024-05-04T18:37:18.492319+0000","172.21.15.32:0/1450308494":"2024-05-04T18:37:18.492319+0000","172.21.15.32:0/3957699240":"2024-05-04T18:37:45.879602+0000","172.21.15.32:0/3310503393":"2024-05-04T18:37:18.492319+0000","172.21.15.32:6801/3945405684":"2024-05-04T18:37:18.492319+0000","172.21.15.32:6800/3945405684":"2024-05-04T18:37:18.492319+0000","172.21.15.32:0/2471164468":"2024-05-04T18:37:45.879602+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-03T18:45:45.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:45.015+0000 7f74a45c3700 1 -- 172.21.15.32:0/3894972247 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f74880b2910 msgr2=0x7f74880b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:45.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:45.015+0000 7f74a45c3700 1 --2- 172.21.15.32:0/3894972247 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f74880b2910 0x7f74880b4dd0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f749c19aa50 tx=0x7f748c009670 comp rx=0 tx=0).stop 2024-05-03T18:45:45.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:45.015+0000 7f74a45c3700 1 -- 172.21.15.32:0/3894972247 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f749c104130 msgr2=0x7f749c19fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:45.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:45.015+0000 7f74a45c3700 1 --2- 172.21.15.32:0/3894972247 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f749c104130 0x7f749c19fc90 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f749800b580 tx=0x7f74980047b0 comp rx=0 tx=0).stop 2024-05-03T18:45:45.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:45.015+0000 7f74a45c3700 1 -- 172.21.15.32:0/3894972247 shutdown_connections 2024-05-03T18:45:45.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:45.015+0000 7f74a45c3700 1 --2- 172.21.15.32:0/3894972247 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f74880b2910 0x7f74880b4dd0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:45.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:45.015+0000 7f74a45c3700 1 --2- 172.21.15.32:0/3894972247 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f749c108870 0x7f749c199910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:45.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:45.015+0000 7f74a45c3700 1 --2- 172.21.15.32:0/3894972247 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f749c103780 0x7f749c19f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:45.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:45.015+0000 7f74a45c3700 1 --2- 172.21.15.32:0/3894972247 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f749c104130 0x7f749c19fc90 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:45.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:45.015+0000 7f74a45c3700 1 -- 172.21.15.32:0/3894972247 >> 172.21.15.32:0/3894972247 conn(0x7f749c07a610 msgr2=0x7f749c101830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:45:45.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:45.015+0000 7f74a45c3700 1 -- 172.21.15.32:0/3894972247 shutdown_connections 2024-05-03T18:45:45.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:45.015+0000 7f74a45c3700 1 -- 172.21.15.32:0/3894972247 wait complete. 2024-05-03T18:45:45.630 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid f500455e-097b-11ef-bc97-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-05-03T18:45:45.630 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid f500455e-097b-11ef-bc97-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-05-03T18:45:45.631 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid f500455e-097b-11ef-bc97-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-05-03T18:45:45.631 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid f500455e-097b-11ef-bc97-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-05-03T18:45:45.632 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid f500455e-097b-11ef-bc97-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-05-03T18:45:45.632 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid f500455e-097b-11ef-bc97-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-05-03T18:45:45.632 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid f500455e-097b-11ef-bc97-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-05-03T18:45:45.633 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid f500455e-097b-11ef-bc97-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-05-03T18:45:45.633 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid f500455e-097b-11ef-bc97-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-05-03T18:45:45.634 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid f500455e-097b-11ef-bc97-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-05-03T18:45:45.634 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid f500455e-097b-11ef-bc97-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-05-03T18:45:45.634 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid f500455e-097b-11ef-bc97-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-05-03T18:45:45.897 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:45:45 smithi113 ceph-mon[33540]: pgmap v307: 97 pgs: 97 active+clean; 631 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-03T18:45:45.898 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:45:45 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:45:45.898 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:45:45 smithi113 ceph-mon[33540]: from='client.? 172.21.15.32:0/3894972247' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T18:45:45.920 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:45:45 smithi045 ceph-mon[33299]: pgmap v307: 97 pgs: 97 active+clean; 631 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-03T18:45:45.921 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:45:45 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:45:45.921 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:45:45 smithi045 ceph-mon[33299]: from='client.? 172.21.15.32:0/3894972247' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T18:45:46.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:45:45 smithi032 ceph-mon[29763]: pgmap v307: 97 pgs: 97 active+clean; 631 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-03T18:45:46.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:45:45 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:45:46.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:45:45 smithi032 ceph-mon[29763]: from='client.? 172.21.15.32:0/3894972247' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T18:45:46.727 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:45:46.728 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:45:46.729 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:45:46.827 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:45:46.873 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:45:46 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]: dispatch 2024-05-03T18:45:46.874 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:45:46 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]: dispatch 2024-05-03T18:45:46.874 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:45:46 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [5, 8]}]: dispatch 2024-05-03T18:45:46.874 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:45:46 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 7]}]: dispatch 2024-05-03T18:45:46.874 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:45:46 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6]}]: dispatch 2024-05-03T18:45:46.874 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:45:46 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 6]}]: dispatch 2024-05-03T18:45:46.874 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:45:46 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2f", "id": [1, 9]}]: dispatch 2024-05-03T18:45:46.874 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:45:46 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 8]}]: dispatch 2024-05-03T18:45:46.874 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:45:46 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]: dispatch 2024-05-03T18:45:46.874 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:45:46 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3b", "id": [3, 9]}]: dispatch 2024-05-03T18:45:46.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:45:46 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]: dispatch 2024-05-03T18:45:46.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:45:46 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]: dispatch 2024-05-03T18:45:46.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:45:46 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [5, 8]}]: dispatch 2024-05-03T18:45:46.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:45:46 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 7]}]: dispatch 2024-05-03T18:45:46.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:45:46 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6]}]: dispatch 2024-05-03T18:45:46.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:45:46 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 6]}]: dispatch 2024-05-03T18:45:46.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:45:46 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2f", "id": [1, 9]}]: dispatch 2024-05-03T18:45:46.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:45:46 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 8]}]: dispatch 2024-05-03T18:45:46.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:45:46 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]: dispatch 2024-05-03T18:45:46.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:45:46 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3b", "id": [3, 9]}]: dispatch 2024-05-03T18:45:47.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:45:46 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]: dispatch 2024-05-03T18:45:47.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:45:46 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]: dispatch 2024-05-03T18:45:47.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:45:46 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [5, 8]}]: dispatch 2024-05-03T18:45:47.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:45:46 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 7]}]: dispatch 2024-05-03T18:45:47.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:45:46 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6]}]: dispatch 2024-05-03T18:45:47.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:45:46 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 6]}]: dispatch 2024-05-03T18:45:47.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:45:46 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2f", "id": [1, 9]}]: dispatch 2024-05-03T18:45:47.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:45:46 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 8]}]: dispatch 2024-05-03T18:45:47.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:45:46 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]: dispatch 2024-05-03T18:45:47.052 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:45:46 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3b", "id": [3, 9]}]: dispatch 2024-05-03T18:45:47.847 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:45:47 smithi032 ceph-mon[29763]: pgmap v308: 97 pgs: 97 active+clean; 631 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:45:47.847 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:45:47 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]': finished 2024-05-03T18:45:47.847 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:45:47 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]': finished 2024-05-03T18:45:47.847 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:45:47 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [5, 8]}]': finished 2024-05-03T18:45:47.847 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:45:47 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 7]}]': finished 2024-05-03T18:45:47.847 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:45:47 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6]}]': finished 2024-05-03T18:45:47.847 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:45:47 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 6]}]': finished 2024-05-03T18:45:47.848 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:45:47 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2f", "id": [1, 9]}]': finished 2024-05-03T18:45:47.848 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:45:47 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 8]}]': finished 2024-05-03T18:45:47.848 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:45:47 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]': finished 2024-05-03T18:45:47.848 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:45:47 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3b", "id": [3, 9]}]': finished 2024-05-03T18:45:47.848 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:45:47 smithi032 ceph-mon[29763]: osdmap e79: 12 total, 12 up, 12 in 2024-05-03T18:45:47.848 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:45:47 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:45:47.848 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:45:47 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:45:47.848 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:45:47 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:45:47.848 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:45:47 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:45:47.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:45:47 smithi045 ceph-mon[33299]: pgmap v308: 97 pgs: 97 active+clean; 631 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:45:47.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:45:47 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]': finished 2024-05-03T18:45:47.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:45:47 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]': finished 2024-05-03T18:45:47.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:45:47 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [5, 8]}]': finished 2024-05-03T18:45:47.999 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:45:47 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 7]}]': finished 2024-05-03T18:45:47.999 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:45:47 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6]}]': finished 2024-05-03T18:45:47.999 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:45:47 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 6]}]': finished 2024-05-03T18:45:47.999 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:45:47 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2f", "id": [1, 9]}]': finished 2024-05-03T18:45:47.999 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:45:47 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 8]}]': finished 2024-05-03T18:45:47.999 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:45:47 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]': finished 2024-05-03T18:45:47.999 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:45:47 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3b", "id": [3, 9]}]': finished 2024-05-03T18:45:47.999 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:45:47 smithi045 ceph-mon[33299]: osdmap e79: 12 total, 12 up, 12 in 2024-05-03T18:45:47.999 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:45:47 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:45:47.999 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:45:47 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:45:47.999 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:45:47 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:45:47.999 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:45:47 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:45:48.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:45:47 smithi113 ceph-mon[33540]: pgmap v308: 97 pgs: 97 active+clean; 631 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:45:48.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:45:47 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]': finished 2024-05-03T18:45:48.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:45:47 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]': finished 2024-05-03T18:45:48.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:45:47 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [5, 8]}]': finished 2024-05-03T18:45:48.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:45:47 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 7]}]': finished 2024-05-03T18:45:48.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:45:47 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6]}]': finished 2024-05-03T18:45:48.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:45:47 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 6]}]': finished 2024-05-03T18:45:48.052 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:45:47 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2f", "id": [1, 9]}]': finished 2024-05-03T18:45:48.052 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:45:47 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 8]}]': finished 2024-05-03T18:45:48.052 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:45:47 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]': finished 2024-05-03T18:45:48.052 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:45:47 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3b", "id": [3, 9]}]': finished 2024-05-03T18:45:48.052 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:45:47 smithi113 ceph-mon[33540]: osdmap e79: 12 total, 12 up, 12 in 2024-05-03T18:45:48.052 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:45:47 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:45:48.052 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:45:47 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:45:48.052 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:45:47 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:45:48.052 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:45:47 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:45:48.395 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:45:48.592 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:45:48.997 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:45:48 smithi045 ceph-mon[33299]: osdmap e80: 12 total, 12 up, 12 in 2024-05-03T18:45:48.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:45:48 smithi045 ceph-mon[33299]: pgmap v311: 97 pgs: 3 peering, 94 active+clean; 631 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-03T18:45:49.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:45:48 smithi113 ceph-mon[33540]: osdmap e80: 12 total, 12 up, 12 in 2024-05-03T18:45:49.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:45:48 smithi113 ceph-mon[33540]: pgmap v311: 97 pgs: 3 peering, 94 active+clean; 631 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-03T18:45:49.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:45:48 smithi032 ceph-mon[29763]: osdmap e80: 12 total, 12 up, 12 in 2024-05-03T18:45:49.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:45:48 smithi032 ceph-mon[29763]: pgmap v311: 97 pgs: 3 peering, 94 active+clean; 631 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-03T18:45:51.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:45:51 smithi113 ceph-mon[33540]: pgmap v312: 97 pgs: 3 peering, 94 active+clean; 631 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s; 0 B/s, 0 objects/s recovering 2024-05-03T18:45:51.311 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:45:51 smithi032 ceph-mon[29763]: pgmap v312: 97 pgs: 3 peering, 94 active+clean; 631 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s; 0 B/s, 0 objects/s recovering 2024-05-03T18:45:51.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:45:51 smithi045 ceph-mon[33299]: pgmap v312: 97 pgs: 3 peering, 94 active+clean; 631 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s; 0 B/s, 0 objects/s recovering 2024-05-03T18:45:52.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:52.007+0000 7f8051c1c700 1 -- 172.21.15.32:0/2805618569 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f804c109590 msgr2=0x7f804c109970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:52.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:52.007+0000 7f8051c1c700 1 --2- 172.21.15.32:0/2805618569 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f804c109590 0x7f804c109970 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f803c0099c0 tx=0x7f803c009cd0 comp rx=0 tx=0).stop 2024-05-03T18:45:52.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:52.007+0000 7f8051c1c700 1 -- 172.21.15.32:0/2805618569 shutdown_connections 2024-05-03T18:45:52.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:52.007+0000 7f8051c1c700 1 --2- 172.21.15.32:0/2805618569 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f804c06c630 0x7f804c110360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:52.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:52.007+0000 7f8051c1c700 1 --2- 172.21.15.32:0/2805618569 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f804c109590 0x7f804c109970 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:52.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:52.007+0000 7f8051c1c700 1 --2- 172.21.15.32:0/2805618569 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f804c06bc70 0x7f804c06c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:52.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:52.007+0000 7f8051c1c700 1 -- 172.21.15.32:0/2805618569 >> 172.21.15.32:0/2805618569 conn(0x7f804c079440 msgr2=0x7f804c079850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:45:52.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:52.007+0000 7f8051c1c700 1 -- 172.21.15.32:0/2805618569 shutdown_connections 2024-05-03T18:45:52.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:52.007+0000 7f8051c1c700 1 -- 172.21.15.32:0/2805618569 wait complete. 2024-05-03T18:45:52.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:52.008+0000 7f8051c1c700 1 Processor -- start 2024-05-03T18:45:52.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:52.008+0000 7f8051c1c700 1 -- start start 2024-05-03T18:45:52.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:52.008+0000 7f8051c1c700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f804c06bc70 0x7f804c19f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:52.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:52.008+0000 7f8051c1c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f804c06c630 0x7f804c19fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:52.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:52.008+0000 7f8051c1c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f804c109590 0x7f804c199960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:52.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:52.008+0000 7f8051c1c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f804c113700 con 0x7f804c06c630 2024-05-03T18:45:52.011 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:52.008+0000 7f8051c1c700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f804c113580 con 0x7f804c109590 2024-05-03T18:45:52.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:52.008+0000 7f8051c1c700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f804c113880 con 0x7f804c06bc70 2024-05-03T18:45:52.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:52.008+0000 7f804affd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f804c06c630 0x7f804c19fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:52.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:52.008+0000 7f804affd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f804c06c630 0x7f804c19fce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44664/0 (socket says 172.21.15.32:44664) 2024-05-03T18:45:52.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:52.008+0000 7f804affd700 1 -- 172.21.15.32:0/1017809579 learned_addr learned my addr 172.21.15.32:0/1017809579 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:45:52.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:52.009+0000 7f804b7fe700 1 --2- 172.21.15.32:0/1017809579 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f804c06bc70 0x7f804c19f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:52.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:52.009+0000 7f804affd700 1 -- 172.21.15.32:0/1017809579 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f804c06bc70 msgr2=0x7f804c19f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:52.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:52.008+0000 7f804bfff700 1 --2- 172.21.15.32:0/1017809579 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f804c109590 0x7f804c199960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:52.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:52.009+0000 7f804affd700 1 --2- 172.21.15.32:0/1017809579 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f804c06bc70 0x7f804c19f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:52.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:52.009+0000 7f804affd700 1 -- 172.21.15.32:0/1017809579 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f804c109590 msgr2=0x7f804c199960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:52.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:52.009+0000 7f804affd700 1 --2- 172.21.15.32:0/1017809579 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f804c109590 0x7f804c199960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:52.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:52.009+0000 7f804affd700 1 -- 172.21.15.32:0/1017809579 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f804c19a0c0 con 0x7f804c06c630 2024-05-03T18:45:52.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:52.009+0000 7f804b7fe700 1 --2- 172.21.15.32:0/1017809579 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f804c06bc70 0x7f804c19f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:45:52.012 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:52.009+0000 7f804bfff700 1 --2- 172.21.15.32:0/1017809579 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f804c109590 0x7f804c199960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:45:52.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:52.009+0000 7f804affd700 1 --2- 172.21.15.32:0/1017809579 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f804c06c630 0x7f804c19fce0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f804000c920 tx=0x7f804000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:45:52.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:52.009+0000 7f8048ff9700 1 -- 172.21.15.32:0/1017809579 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8040007b00 con 0x7f804c06c630 2024-05-03T18:45:52.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:52.009+0000 7f8051c1c700 1 -- 172.21.15.32:0/1017809579 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f804c19a3b0 con 0x7f804c06c630 2024-05-03T18:45:52.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:52.009+0000 7f8048ff9700 1 -- 172.21.15.32:0/1017809579 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8040004ce0 con 0x7f804c06c630 2024-05-03T18:45:52.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:52.009+0000 7f8048ff9700 1 -- 172.21.15.32:0/1017809579 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8040005650 con 0x7f804c06c630 2024-05-03T18:45:52.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:52.009+0000 7f8051c1c700 1 -- 172.21.15.32:0/1017809579 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f804c1a6670 con 0x7f804c06c630 2024-05-03T18:45:52.013 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:52.010+0000 7f8051c1c700 1 -- 172.21.15.32:0/1017809579 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f804c063c30 con 0x7f804c06c630 2024-05-03T18:45:52.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:52.012+0000 7f8048ff9700 1 -- 172.21.15.32:0/1017809579 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f8040004750 con 0x7f804c06c630 2024-05-03T18:45:52.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:52.013+0000 7f8048ff9700 1 --2- 172.21.15.32:0/1017809579 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f80340b2910 0x7f80340b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:52.014 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:52.013+0000 7f804b7fe700 1 --2- 172.21.15.32:0/1017809579 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f80340b2910 0x7f80340b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:52.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:52.013+0000 7f8048ff9700 1 -- 172.21.15.32:0/1017809579 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8269+0+0 (secure 0 0 0) 0x7f80400e25f0 con 0x7f804c06c630 2024-05-03T18:45:52.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:52.013+0000 7f8048ff9700 1 --2- 172.21.15.32:0/1017809579 >> [v2:172.21.15.45:6824/1817064361,v1:172.21.15.45:6825/1817064361] conn(0x7f80340bc550 0x7f80340be9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:52.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:52.013+0000 7f8048ff9700 1 -- 172.21.15.32:0/1017809579 --> [v2:172.21.15.45:6824/1817064361,v1:172.21.15.45:6825/1817064361] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8040012ec0 con 0x7f80340bc550 2024-05-03T18:45:52.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:52.013+0000 7f8048ff9700 1 -- 172.21.15.32:0/1017809579 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7f80400e29a0 con 0x7f804c06c630 2024-05-03T18:45:52.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:52.014+0000 7f804b7fe700 1 --2- 172.21.15.32:0/1017809579 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f80340b2910 0x7f80340b4dd0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f803c00b580 tx=0x7f803c009f10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:45:52.015 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:52.014+0000 7f804bfff700 1 --2- 172.21.15.32:0/1017809579 >> [v2:172.21.15.45:6824/1817064361,v1:172.21.15.45:6825/1817064361] conn(0x7f80340bc550 0x7f80340be9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:52.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:52.014+0000 7f804bfff700 1 --2- 172.21.15.32:0/1017809579 >> [v2:172.21.15.45:6824/1817064361,v1:172.21.15.45:6825/1817064361] conn(0x7f80340bc550 0x7f80340be9f0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:45:52.017 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:52.015+0000 7f8048ff9700 1 -- 172.21.15.32:0/1017809579 <== osd.10 v2:172.21.15.45:6824/1817064361 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8040012ec0 con 0x7f80340bc550 2024-05-03T18:45:52.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:52.032+0000 7f8051c1c700 1 -- 172.21.15.32:0/1017809579 --> [v2:172.21.15.45:6824/1817064361,v1:172.21.15.45:6825/1817064361] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f804c063ec0 con 0x7f80340bc550 2024-05-03T18:45:52.033 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:52.032+0000 7f8048ff9700 1 -- 172.21.15.32:0/1017809579 <== osd.10 v2:172.21.15.45:6824/1817064361 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f804c063ec0 con 0x7f80340bc550 2024-05-03T18:45:52.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:52.033+0000 7f8051c1c700 1 -- 172.21.15.32:0/1017809579 >> [v2:172.21.15.45:6824/1817064361,v1:172.21.15.45:6825/1817064361] conn(0x7f80340bc550 msgr2=0x7f80340be9f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:52.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:52.033+0000 7f8051c1c700 1 --2- 172.21.15.32:0/1017809579 >> [v2:172.21.15.45:6824/1817064361,v1:172.21.15.45:6825/1817064361] conn(0x7f80340bc550 0x7f80340be9f0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:52.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:52.033+0000 7f8051c1c700 1 -- 172.21.15.32:0/1017809579 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f80340b2910 msgr2=0x7f80340b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:52.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:52.033+0000 7f8051c1c700 1 --2- 172.21.15.32:0/1017809579 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f80340b2910 0x7f80340b4dd0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f803c00b580 tx=0x7f803c009f10 comp rx=0 tx=0).stop 2024-05-03T18:45:52.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:52.033+0000 7f8051c1c700 1 -- 172.21.15.32:0/1017809579 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f804c06c630 msgr2=0x7f804c19fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:52.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:52.033+0000 7f8051c1c700 1 --2- 172.21.15.32:0/1017809579 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f804c06c630 0x7f804c19fce0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f804000c920 tx=0x7f804000cce0 comp rx=0 tx=0).stop 2024-05-03T18:45:52.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:52.033+0000 7f804bfff700 1 -- 172.21.15.32:0/1017809579 reap_dead start 2024-05-03T18:45:52.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:52.033+0000 7f8051c1c700 1 -- 172.21.15.32:0/1017809579 shutdown_connections 2024-05-03T18:45:52.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:52.033+0000 7f8051c1c700 1 -- 172.21.15.32:0/1017809579 >> 172.21.15.32:0/1017809579 conn(0x7f804c079440 msgr2=0x7f804c100f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:45:52.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:52.033+0000 7f8051c1c700 1 -- 172.21.15.32:0/1017809579 shutdown_connections 2024-05-03T18:45:52.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:52.033+0000 7f8051c1c700 1 -- 172.21.15.32:0/1017809579 wait complete. 2024-05-03T18:45:53.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:45:53 smithi113 ceph-mon[33540]: pgmap v313: 97 pgs: 1 peering, 96 active+clean; 631 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s; 14 B/s, 0 objects/s recovering 2024-05-03T18:45:53.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:45:53 smithi032 ceph-mon[29763]: pgmap v313: 97 pgs: 1 peering, 96 active+clean; 631 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s; 14 B/s, 0 objects/s recovering 2024-05-03T18:45:53.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:45:53 smithi045 ceph-mon[33299]: pgmap v313: 97 pgs: 1 peering, 96 active+clean; 631 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s; 14 B/s, 0 objects/s recovering 2024-05-03T18:45:54.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:54.039+0000 7f5c38185700 1 -- 172.21.15.32:0/3949306559 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c30109590 msgr2=0x7f5c30109970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:54.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:54.039+0000 7f5c38185700 1 --2- 172.21.15.32:0/3949306559 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c30109590 0x7f5c30109970 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f5c2c0099c0 tx=0x7f5c2c009cd0 comp rx=0 tx=0).stop 2024-05-03T18:45:54.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:54.039+0000 7f5c38185700 1 -- 172.21.15.32:0/3949306559 shutdown_connections 2024-05-03T18:45:54.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:54.039+0000 7f5c38185700 1 --2- 172.21.15.32:0/3949306559 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5c3006c630 0x7f5c30110360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:54.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:54.039+0000 7f5c38185700 1 --2- 172.21.15.32:0/3949306559 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c30109590 0x7f5c30109970 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:54.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:54.039+0000 7f5c38185700 1 --2- 172.21.15.32:0/3949306559 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c3006bc70 0x7f5c3006c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:54.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:54.039+0000 7f5c38185700 1 -- 172.21.15.32:0/3949306559 >> 172.21.15.32:0/3949306559 conn(0x7f5c30079440 msgr2=0x7f5c30079850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:45:54.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:54.040+0000 7f5c38185700 1 -- 172.21.15.32:0/3949306559 shutdown_connections 2024-05-03T18:45:54.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:54.040+0000 7f5c38185700 1 -- 172.21.15.32:0/3949306559 wait complete. 2024-05-03T18:45:54.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:54.040+0000 7f5c38185700 1 Processor -- start 2024-05-03T18:45:54.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:54.040+0000 7f5c38185700 1 -- start start 2024-05-03T18:45:54.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:54.040+0000 7f5c38185700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c3006bc70 0x7f5c3019f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:54.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:54.040+0000 7f5c38185700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5c3006c630 0x7f5c3019fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:54.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:54.040+0000 7f5c38185700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c30109590 0x7f5c30199900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:54.043 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:54.040+0000 7f5c38185700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f5c30113680 con 0x7f5c3006bc70 2024-05-03T18:45:54.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:54.040+0000 7f5c38185700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f5c30113500 con 0x7f5c30109590 2024-05-03T18:45:54.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:54.040+0000 7f5c38185700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f5c30113800 con 0x7f5c3006c630 2024-05-03T18:45:54.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:54.041+0000 7f5c35f21700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c3006bc70 0x7f5c3019f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:54.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:54.041+0000 7f5c35720700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5c3006c630 0x7f5c3019fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:54.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:54.041+0000 7f5c35f21700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c3006bc70 0x7f5c3019f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53132/0 (socket says 172.21.15.32:53132) 2024-05-03T18:45:54.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:54.041+0000 7f5c35f21700 1 -- 172.21.15.32:0/1344481898 learned_addr learned my addr 172.21.15.32:0/1344481898 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:45:54.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:54.041+0000 7f5c36722700 1 --2- 172.21.15.32:0/1344481898 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c30109590 0x7f5c30199900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:54.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:54.041+0000 7f5c35f21700 1 -- 172.21.15.32:0/1344481898 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5c3006c630 msgr2=0x7f5c3019fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:54.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:54.041+0000 7f5c35f21700 1 --2- 172.21.15.32:0/1344481898 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5c3006c630 0x7f5c3019fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:54.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:54.041+0000 7f5c35f21700 1 -- 172.21.15.32:0/1344481898 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c30109590 msgr2=0x7f5c30199900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:54.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:54.041+0000 7f5c35f21700 1 --2- 172.21.15.32:0/1344481898 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c30109590 0x7f5c30199900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:54.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:54.041+0000 7f5c35f21700 1 -- 172.21.15.32:0/1344481898 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c3019a060 con 0x7f5c3006bc70 2024-05-03T18:45:54.044 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:54.041+0000 7f5c35f21700 1 --2- 172.21.15.32:0/1344481898 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c3006bc70 0x7f5c3019f740 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f5c2c00b580 tx=0x7f5c2c0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:45:54.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:54.041+0000 7f5c26ffd700 1 -- 172.21.15.32:0/1344481898 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c2c027070 con 0x7f5c3006bc70 2024-05-03T18:45:54.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:54.041+0000 7f5c26ffd700 1 -- 172.21.15.32:0/1344481898 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5c2c003f20 con 0x7f5c3006bc70 2024-05-03T18:45:54.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:54.041+0000 7f5c38185700 1 -- 172.21.15.32:0/1344481898 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c3019a2f0 con 0x7f5c3006bc70 2024-05-03T18:45:54.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:54.041+0000 7f5c26ffd700 1 -- 172.21.15.32:0/1344481898 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c2c022d10 con 0x7f5c3006bc70 2024-05-03T18:45:54.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:54.042+0000 7f5c38185700 1 -- 172.21.15.32:0/1344481898 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c301a6610 con 0x7f5c3006bc70 2024-05-03T18:45:54.045 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:54.042+0000 7f5c38185700 1 -- 172.21.15.32:0/1344481898 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5c30063bb0 con 0x7f5c3006bc70 2024-05-03T18:45:54.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:54.044+0000 7f5c26ffd700 1 -- 172.21.15.32:0/1344481898 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f5c2c022470 con 0x7f5c3006bc70 2024-05-03T18:45:54.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:54.044+0000 7f5c26ffd700 1 --2- 172.21.15.32:0/1344481898 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f5c1c0b2990 0x7f5c1c0b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:54.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:54.045+0000 7f5c35720700 1 --2- 172.21.15.32:0/1344481898 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f5c1c0b2990 0x7f5c1c0b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:54.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:54.045+0000 7f5c26ffd700 1 -- 172.21.15.32:0/1344481898 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8269+0+0 (secure 0 0 0) 0x7f5c2c0e35b0 con 0x7f5c3006bc70 2024-05-03T18:45:54.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:54.045+0000 7f5c26ffd700 1 --2- 172.21.15.32:0/1344481898 >> [v2:172.21.15.45:6800/1209907302,v1:172.21.15.45:6801/1209907302] conn(0x7f5c1c0bc4a0 0x7f5c1c0be940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:54.046 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:54.045+0000 7f5c26ffd700 1 -- 172.21.15.32:0/1344481898 --> [v2:172.21.15.45:6800/1209907302,v1:172.21.15.45:6801/1209907302] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5c2c0043d0 con 0x7f5c1c0bc4a0 2024-05-03T18:45:54.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:54.045+0000 7f5c26ffd700 1 -- 172.21.15.32:0/1344481898 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7f5c2c0e3960 con 0x7f5c3006bc70 2024-05-03T18:45:54.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:54.045+0000 7f5c35720700 1 --2- 172.21.15.32:0/1344481898 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f5c1c0b2990 0x7f5c1c0b4e50 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f5c30108260 tx=0x7f5c2000a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:45:54.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:54.045+0000 7f5c36722700 1 --2- 172.21.15.32:0/1344481898 >> [v2:172.21.15.45:6800/1209907302,v1:172.21.15.45:6801/1209907302] conn(0x7f5c1c0bc4a0 0x7f5c1c0be940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:54.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:54.046+0000 7f5c36722700 1 --2- 172.21.15.32:0/1344481898 >> [v2:172.21.15.45:6800/1209907302,v1:172.21.15.45:6801/1209907302] conn(0x7f5c1c0bc4a0 0x7f5c1c0be940 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:45:54.047 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:54.046+0000 7f5c26ffd700 1 -- 172.21.15.32:0/1344481898 <== osd.1 v2:172.21.15.45:6800/1209907302 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5c2c0043d0 con 0x7f5c1c0bc4a0 2024-05-03T18:45:54.066 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:45:54.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:54.066+0000 7f5c38185700 1 -- 172.21.15.32:0/1344481898 --> [v2:172.21.15.45:6800/1209907302,v1:172.21.15.45:6801/1209907302] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5c30063fb0 con 0x7f5c1c0bc4a0 2024-05-03T18:45:54.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:54.067+0000 7f5c26ffd700 1 -- 172.21.15.32:0/1344481898 <== osd.1 v2:172.21.15.45:6800/1209907302 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f5c30063fb0 con 0x7f5c1c0bc4a0 2024-05-03T18:45:54.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:54.069+0000 7f5c38185700 1 -- 172.21.15.32:0/1344481898 >> [v2:172.21.15.45:6800/1209907302,v1:172.21.15.45:6801/1209907302] conn(0x7f5c1c0bc4a0 msgr2=0x7f5c1c0be940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:54.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:54.069+0000 7f5c38185700 1 --2- 172.21.15.32:0/1344481898 >> [v2:172.21.15.45:6800/1209907302,v1:172.21.15.45:6801/1209907302] conn(0x7f5c1c0bc4a0 0x7f5c1c0be940 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:54.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:54.069+0000 7f5c38185700 1 -- 172.21.15.32:0/1344481898 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f5c1c0b2990 msgr2=0x7f5c1c0b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:54.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:54.069+0000 7f5c38185700 1 --2- 172.21.15.32:0/1344481898 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f5c1c0b2990 0x7f5c1c0b4e50 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f5c30108260 tx=0x7f5c2000a380 comp rx=0 tx=0).stop 2024-05-03T18:45:54.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:54.069+0000 7f5c38185700 1 -- 172.21.15.32:0/1344481898 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c3006bc70 msgr2=0x7f5c3019f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:54.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:54.069+0000 7f5c38185700 1 --2- 172.21.15.32:0/1344481898 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c3006bc70 0x7f5c3019f740 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f5c2c00b580 tx=0x7f5c2c0047b0 comp rx=0 tx=0).stop 2024-05-03T18:45:54.071 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:54.069+0000 7f5c36722700 1 -- 172.21.15.32:0/1344481898 reap_dead start 2024-05-03T18:45:54.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:54.073+0000 7f5c38185700 1 -- 172.21.15.32:0/1344481898 shutdown_connections 2024-05-03T18:45:54.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:54.073+0000 7f5c38185700 1 -- 172.21.15.32:0/1344481898 >> 172.21.15.32:0/1344481898 conn(0x7f5c30079440 msgr2=0x7f5c30100df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:45:54.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:54.073+0000 7f5c38185700 1 -- 172.21.15.32:0/1344481898 shutdown_connections 2024-05-03T18:45:54.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:54.074+0000 7f5c38185700 1 -- 172.21.15.32:0/1344481898 wait complete. 2024-05-03T18:45:54.813 INFO:teuthology.orchestra.run.smithi032.stdout:55834574925 2024-05-03T18:45:54.813 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid f500455e-097b-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-05-03T18:45:55.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:45:55 smithi113 ceph-mon[33540]: pgmap v314: 97 pgs: 97 active+clean; 631 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s; 14 B/s, 0 objects/s recovering 2024-05-03T18:45:55.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:45:55 smithi032 ceph-mon[29763]: pgmap v314: 97 pgs: 97 active+clean; 631 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s; 14 B/s, 0 objects/s recovering 2024-05-03T18:45:55.331 INFO:teuthology.orchestra.run.smithi032.stdout:274877906976 2024-05-03T18:45:55.331 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid f500455e-097b-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-05-03T18:45:55.497 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:45:55 smithi045 ceph-mon[33299]: pgmap v314: 97 pgs: 97 active+clean; 631 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s; 14 B/s, 0 objects/s recovering 2024-05-03T18:45:56.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:56.258+0000 7facd408f700 1 -- 172.21.15.32:0/2055004374 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faccc108410 msgr2=0x7faccc1087f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:56.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:56.258+0000 7facd408f700 1 --2- 172.21.15.32:0/2055004374 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faccc108410 0x7faccc1087f0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7facc4009990 tx=0x7facc4009ca0 comp rx=0 tx=0).stop 2024-05-03T18:45:56.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:56.258+0000 7facd408f700 1 -- 172.21.15.32:0/2055004374 shutdown_connections 2024-05-03T18:45:56.260 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:56.258+0000 7facd408f700 1 --2- 172.21.15.32:0/2055004374 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faccc102d60 0x7faccc10f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:56.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:56.258+0000 7facd408f700 1 --2- 172.21.15.32:0/2055004374 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faccc108410 0x7faccc1087f0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:56.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:56.258+0000 7facd408f700 1 --2- 172.21.15.32:0/2055004374 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faccc1023a0 0x7faccc102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:56.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:56.258+0000 7facd408f700 1 -- 172.21.15.32:0/2055004374 >> 172.21.15.32:0/2055004374 conn(0x7faccc079350 msgr2=0x7faccc079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:45:56.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:56.258+0000 7facd408f700 1 -- 172.21.15.32:0/2055004374 shutdown_connections 2024-05-03T18:45:56.261 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:56.258+0000 7facd408f700 1 -- 172.21.15.32:0/2055004374 wait complete. 2024-05-03T18:45:56.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:56.259+0000 7facd408f700 1 Processor -- start 2024-05-03T18:45:56.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:56.259+0000 7facd408f700 1 -- start start 2024-05-03T18:45:56.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:56.259+0000 7facd408f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faccc1023a0 0x7faccc19f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:56.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:56.259+0000 7facd408f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faccc102d60 0x7faccc19fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:56.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:56.259+0000 7facd408f700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faccc108410 0x7faccc199930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:56.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:56.259+0000 7facd408f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7faccc112530 con 0x7faccc1023a0 2024-05-03T18:45:56.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:56.259+0000 7facd408f700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7faccc1123b0 con 0x7faccc102d60 2024-05-03T18:45:56.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:56.259+0000 7facd408f700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7faccc1126b0 con 0x7faccc108410 2024-05-03T18:45:56.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:56.259+0000 7facd1e2b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faccc1023a0 0x7faccc19f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:56.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:56.259+0000 7facd162a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faccc102d60 0x7faccc19fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:56.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:56.259+0000 7facd262c700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faccc108410 0x7faccc199930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:56.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:56.259+0000 7facd1e2b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faccc1023a0 0x7faccc19f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53154/0 (socket says 172.21.15.32:53154) 2024-05-03T18:45:56.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:56.259+0000 7facd1e2b700 1 -- 172.21.15.32:0/3997877104 learned_addr learned my addr 172.21.15.32:0/3997877104 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:45:56.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:56.259+0000 7facd1e2b700 1 -- 172.21.15.32:0/3997877104 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faccc108410 msgr2=0x7faccc199930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:56.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:56.259+0000 7facd1e2b700 1 --2- 172.21.15.32:0/3997877104 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faccc108410 0x7faccc199930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:56.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:56.259+0000 7facd1e2b700 1 -- 172.21.15.32:0/3997877104 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faccc102d60 msgr2=0x7faccc19fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:56.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:56.259+0000 7facd1e2b700 1 --2- 172.21.15.32:0/3997877104 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faccc102d60 0x7faccc19fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:56.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:56.259+0000 7facd1e2b700 1 -- 172.21.15.32:0/3997877104 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faccc19a090 con 0x7faccc1023a0 2024-05-03T18:45:56.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:56.259+0000 7facd262c700 1 --2- 172.21.15.32:0/3997877104 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faccc108410 0x7faccc199930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T18:45:56.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:56.260+0000 7facd1e2b700 1 --2- 172.21.15.32:0/3997877104 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faccc1023a0 0x7faccc19f770 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7facc4000c00 tx=0x7facc4009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:45:56.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:56.260+0000 7faccaffd700 1 -- 172.21.15.32:0/3997877104 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facc4027070 con 0x7faccc1023a0 2024-05-03T18:45:56.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:56.260+0000 7facd408f700 1 -- 172.21.15.32:0/3997877104 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faccc19a320 con 0x7faccc1023a0 2024-05-03T18:45:56.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:56.260+0000 7faccaffd700 1 -- 172.21.15.32:0/3997877104 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7facc4022470 con 0x7faccc1023a0 2024-05-03T18:45:56.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:56.260+0000 7faccaffd700 1 -- 172.21.15.32:0/3997877104 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facc4017650 con 0x7faccc1023a0 2024-05-03T18:45:56.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:56.260+0000 7facd408f700 1 -- 172.21.15.32:0/3997877104 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faccc1a6640 con 0x7faccc1023a0 2024-05-03T18:45:56.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:56.260+0000 7facd408f700 1 -- 172.21.15.32:0/3997877104 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7faccc107520 con 0x7faccc1023a0 2024-05-03T18:45:56.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:56.262+0000 7faccaffd700 1 -- 172.21.15.32:0/3997877104 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7facc402a070 con 0x7faccc1023a0 2024-05-03T18:45:56.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:56.263+0000 7faccaffd700 1 --2- 172.21.15.32:0/3997877104 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7facb40b2910 0x7facb40b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:56.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:56.263+0000 7facd162a700 1 --2- 172.21.15.32:0/3997877104 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7facb40b2910 0x7facb40b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:56.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:56.263+0000 7faccaffd700 1 -- 172.21.15.32:0/3997877104 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8269+0+0 (secure 0 0 0) 0x7facc40e38d0 con 0x7faccc1023a0 2024-05-03T18:45:56.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:56.263+0000 7faccaffd700 1 --2- 172.21.15.32:0/3997877104 >> [v2:172.21.15.45:6816/1032572438,v1:172.21.15.45:6817/1032572438] conn(0x7facb40bc420 0x7facb40be8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:56.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:56.263+0000 7faccaffd700 1 -- 172.21.15.32:0/3997877104 --> [v2:172.21.15.45:6816/1032572438,v1:172.21.15.45:6817/1032572438] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7facb40bef90 con 0x7facb40bc420 2024-05-03T18:45:56.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:56.263+0000 7faccaffd700 1 -- 172.21.15.32:0/3997877104 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7facc40e3d20 con 0x7faccc1023a0 2024-05-03T18:45:56.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:56.263+0000 7facd162a700 1 --2- 172.21.15.32:0/3997877104 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7facb40b2910 0x7facb40b4dd0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7faccc079850 tx=0x7facc000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:45:56.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:56.263+0000 7facd262c700 1 --2- 172.21.15.32:0/3997877104 >> [v2:172.21.15.45:6816/1032572438,v1:172.21.15.45:6817/1032572438] conn(0x7facb40bc420 0x7facb40be8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:56.266 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:56.264+0000 7facd262c700 1 --2- 172.21.15.32:0/3997877104 >> [v2:172.21.15.45:6816/1032572438,v1:172.21.15.45:6817/1032572438] conn(0x7facb40bc420 0x7facb40be8c0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:45:56.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:56.265+0000 7faccaffd700 1 -- 172.21.15.32:0/3997877104 <== osd.7 v2:172.21.15.45:6816/1032572438 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7facb40bef90 con 0x7facb40bc420 2024-05-03T18:45:56.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:56.281+0000 7facd408f700 1 -- 172.21.15.32:0/3997877104 --> [v2:172.21.15.45:6816/1032572438,v1:172.21.15.45:6817/1032572438] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7faccc103ac0 con 0x7facb40bc420 2024-05-03T18:45:56.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:56.282+0000 7faccaffd700 1 -- 172.21.15.32:0/3997877104 <== osd.7 v2:172.21.15.45:6816/1032572438 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7faccc103ac0 con 0x7facb40bc420 2024-05-03T18:45:56.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:56.282+0000 7facd408f700 1 -- 172.21.15.32:0/3997877104 >> [v2:172.21.15.45:6816/1032572438,v1:172.21.15.45:6817/1032572438] conn(0x7facb40bc420 msgr2=0x7facb40be8c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:56.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:56.283+0000 7facd408f700 1 --2- 172.21.15.32:0/3997877104 >> [v2:172.21.15.45:6816/1032572438,v1:172.21.15.45:6817/1032572438] conn(0x7facb40bc420 0x7facb40be8c0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:56.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:56.283+0000 7facd408f700 1 -- 172.21.15.32:0/3997877104 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7facb40b2910 msgr2=0x7facb40b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:56.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:56.283+0000 7facd408f700 1 --2- 172.21.15.32:0/3997877104 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7facb40b2910 0x7facb40b4dd0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7faccc079850 tx=0x7facc000b410 comp rx=0 tx=0).stop 2024-05-03T18:45:56.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:56.283+0000 7facd408f700 1 -- 172.21.15.32:0/3997877104 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faccc1023a0 msgr2=0x7faccc19f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:56.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:56.283+0000 7facd408f700 1 --2- 172.21.15.32:0/3997877104 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faccc1023a0 0x7faccc19f770 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7facc4000c00 tx=0x7facc4009580 comp rx=0 tx=0).stop 2024-05-03T18:45:56.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:56.283+0000 7facd262c700 1 -- 172.21.15.32:0/3997877104 reap_dead start 2024-05-03T18:45:56.285 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:56.283+0000 7facd408f700 1 -- 172.21.15.32:0/3997877104 shutdown_connections 2024-05-03T18:45:56.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:56.283+0000 7facd408f700 1 -- 172.21.15.32:0/3997877104 >> 172.21.15.32:0/3997877104 conn(0x7faccc079350 msgr2=0x7faccc06b2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:45:56.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:56.283+0000 7facd408f700 1 -- 172.21.15.32:0/3997877104 shutdown_connections 2024-05-03T18:45:56.286 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:56.283+0000 7facd408f700 1 -- 172.21.15.32:0/3997877104 wait complete. 2024-05-03T18:45:56.987 INFO:teuthology.orchestra.run.smithi032.stdout:201863462961 2024-05-03T18:45:56.987 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid f500455e-097b-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-05-03T18:45:57.040 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:45:57 smithi032 ceph-mon[29763]: pgmap v315: 97 pgs: 97 active+clean; 631 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s; 12 B/s, 0 objects/s recovering 2024-05-03T18:45:57.200 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:56.474+0000 7fcbc75f9700 1 -- 172.21.15.32:0/455707958 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcbc01095e0 msgr2=0x7fcbc01099c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:57.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:56.474+0000 7fcbc75f9700 1 --2- 172.21.15.32:0/455707958 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcbc01095e0 0x7fcbc01099c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fcbb80099c0 tx=0x7fcbb8009cd0 comp rx=0 tx=0).stop 2024-05-03T18:45:57.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:56.474+0000 7fcbc75f9700 1 -- 172.21.15.32:0/455707958 shutdown_connections 2024-05-03T18:45:57.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:56.474+0000 7fcbc75f9700 1 --2- 172.21.15.32:0/455707958 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcbc0103f00 0x7fcbc01103b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:57.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:56.474+0000 7fcbc75f9700 1 --2- 172.21.15.32:0/455707958 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcbc01095e0 0x7fcbc01099c0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:57.201 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:56.474+0000 7fcbc75f9700 1 --2- 172.21.15.32:0/455707958 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcbc0103540 0x7fcbc01039c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:57.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:56.474+0000 7fcbc75f9700 1 -- 172.21.15.32:0/455707958 >> 172.21.15.32:0/455707958 conn(0x7fcbc007a650 msgr2=0x7fcbc007aa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:45:57.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:56.474+0000 7fcbc75f9700 1 -- 172.21.15.32:0/455707958 shutdown_connections 2024-05-03T18:45:57.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:56.475+0000 7fcbc75f9700 1 -- 172.21.15.32:0/455707958 wait complete. 2024-05-03T18:45:57.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:56.475+0000 7fcbc75f9700 1 Processor -- start 2024-05-03T18:45:57.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:56.475+0000 7fcbc75f9700 1 -- start start 2024-05-03T18:45:57.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:56.475+0000 7fcbc75f9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcbc0103540 0x7fcbc00757a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:57.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:56.475+0000 7fcbc75f9700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcbc0103f00 0x7fcbc0075ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:57.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:56.475+0000 7fcbc75f9700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcbc01095e0 0x7fcbc0076220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:57.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:56.476+0000 7fcbc75f9700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fcbc01136e0 con 0x7fcbc0103540 2024-05-03T18:45:57.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:56.476+0000 7fcbc75f9700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fcbc0113560 con 0x7fcbc01095e0 2024-05-03T18:45:57.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:56.476+0000 7fcbc75f9700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fcbc0113860 con 0x7fcbc0103f00 2024-05-03T18:45:57.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:56.476+0000 7fcbc5395700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcbc0103540 0x7fcbc00757a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:57.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:56.476+0000 7fcbc5395700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcbc0103540 0x7fcbc00757a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53164/0 (socket says 172.21.15.32:53164) 2024-05-03T18:45:57.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:56.476+0000 7fcbc5395700 1 -- 172.21.15.32:0/3979193053 learned_addr learned my addr 172.21.15.32:0/3979193053 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:45:57.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:56.476+0000 7fcbc4b94700 1 --2- 172.21.15.32:0/3979193053 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcbc0103f00 0x7fcbc0075ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:57.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:56.476+0000 7fcbc5b96700 1 --2- 172.21.15.32:0/3979193053 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcbc01095e0 0x7fcbc0076220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:57.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:56.476+0000 7fcbc5395700 1 -- 172.21.15.32:0/3979193053 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcbc0103f00 msgr2=0x7fcbc0075ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:57.204 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:56.476+0000 7fcbc5395700 1 --2- 172.21.15.32:0/3979193053 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcbc0103f00 0x7fcbc0075ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:57.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:56.476+0000 7fcbc5395700 1 -- 172.21.15.32:0/3979193053 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcbc01095e0 msgr2=0x7fcbc0076220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:57.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:56.476+0000 7fcbc5395700 1 --2- 172.21.15.32:0/3979193053 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcbc01095e0 0x7fcbc0076220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:57.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:56.476+0000 7fcbc5395700 1 -- 172.21.15.32:0/3979193053 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbc01a80c0 con 0x7fcbc0103540 2024-05-03T18:45:57.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:56.476+0000 7fcbc5395700 1 --2- 172.21.15.32:0/3979193053 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcbc0103540 0x7fcbc00757a0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fcbb800b580 tx=0x7fcbb80047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:45:57.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:56.476+0000 7fcbbe7fc700 1 -- 172.21.15.32:0/3979193053 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbb8027070 con 0x7fcbc0103540 2024-05-03T18:45:57.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:56.477+0000 7fcbc75f9700 1 -- 172.21.15.32:0/3979193053 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbc01a8350 con 0x7fcbc0103540 2024-05-03T18:45:57.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:56.477+0000 7fcbbe7fc700 1 -- 172.21.15.32:0/3979193053 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcbb8004020 con 0x7fcbc0103540 2024-05-03T18:45:57.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:56.477+0000 7fcbc75f9700 1 -- 172.21.15.32:0/3979193053 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbc01a8800 con 0x7fcbc0103540 2024-05-03T18:45:57.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:56.477+0000 7fcbbe7fc700 1 -- 172.21.15.32:0/3979193053 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbb8022e10 con 0x7fcbc0103540 2024-05-03T18:45:57.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:56.478+0000 7fcbc75f9700 1 -- 172.21.15.32:0/3979193053 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fcbac001030 con 0x7fcbc0103540 2024-05-03T18:45:57.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:56.479+0000 7fcbbe7fc700 1 -- 172.21.15.32:0/3979193053 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fcbb802a070 con 0x7fcbc0103540 2024-05-03T18:45:57.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:56.480+0000 7fcbbe7fc700 1 --2- 172.21.15.32:0/3979193053 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fcba80b2910 0x7fcba80b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:57.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:56.480+0000 7fcbc4b94700 1 --2- 172.21.15.32:0/3979193053 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fcba80b2910 0x7fcba80b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:57.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:56.480+0000 7fcbbe7fc700 1 -- 172.21.15.32:0/3979193053 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8269+0+0 (secure 0 0 0) 0x7fcbb80e3d60 con 0x7fcbc0103540 2024-05-03T18:45:57.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:56.480+0000 7fcbbe7fc700 1 --2- 172.21.15.32:0/3979193053 >> [v2:172.21.15.32:6818/791407796,v1:172.21.15.32:6819/791407796] conn(0x7fcba80bc550 0x7fcba80be9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:57.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:56.480+0000 7fcbbe7fc700 1 -- 172.21.15.32:0/3979193053 --> [v2:172.21.15.32:6818/791407796,v1:172.21.15.32:6819/791407796] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fcbb8014e50 con 0x7fcba80bc550 2024-05-03T18:45:57.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:56.480+0000 7fcbbe7fc700 1 -- 172.21.15.32:0/3979193053 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7fcbb80e41b0 con 0x7fcbc0103540 2024-05-03T18:45:57.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:56.480+0000 7fcbc4b94700 1 --2- 172.21.15.32:0/3979193053 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fcba80b2910 0x7fcba80b4dd0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fcbc007acc0 tx=0x7fcbb400a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:45:57.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:56.480+0000 7fcbc5b96700 1 --2- 172.21.15.32:0/3979193053 >> [v2:172.21.15.32:6818/791407796,v1:172.21.15.32:6819/791407796] conn(0x7fcba80bc550 0x7fcba80be9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:57.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:56.481+0000 7fcbc5b96700 1 --2- 172.21.15.32:0/3979193053 >> [v2:172.21.15.32:6818/791407796,v1:172.21.15.32:6819/791407796] conn(0x7fcba80bc550 0x7fcba80be9f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:45:57.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:56.482+0000 7fcbbe7fc700 1 -- 172.21.15.32:0/3979193053 <== osd.6 v2:172.21.15.32:6818/791407796 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fcbb8014e50 con 0x7fcba80bc550 2024-05-03T18:45:57.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:56.498+0000 7fcbc75f9700 1 -- 172.21.15.32:0/3979193053 --> [v2:172.21.15.32:6818/791407796,v1:172.21.15.32:6819/791407796] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fcbac002e60 con 0x7fcba80bc550 2024-05-03T18:45:57.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:56.499+0000 7fcbbe7fc700 1 -- 172.21.15.32:0/3979193053 <== osd.6 v2:172.21.15.32:6818/791407796 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fcbac002e60 con 0x7fcba80bc550 2024-05-03T18:45:57.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:56.499+0000 7fcbc75f9700 1 -- 172.21.15.32:0/3979193053 >> [v2:172.21.15.32:6818/791407796,v1:172.21.15.32:6819/791407796] conn(0x7fcba80bc550 msgr2=0x7fcba80be9f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:57.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:56.499+0000 7fcbc75f9700 1 --2- 172.21.15.32:0/3979193053 >> [v2:172.21.15.32:6818/791407796,v1:172.21.15.32:6819/791407796] conn(0x7fcba80bc550 0x7fcba80be9f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:57.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:56.499+0000 7fcbc75f9700 1 -- 172.21.15.32:0/3979193053 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fcba80b2910 msgr2=0x7fcba80b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:57.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:56.499+0000 7fcbc75f9700 1 --2- 172.21.15.32:0/3979193053 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fcba80b2910 0x7fcba80b4dd0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fcbc007acc0 tx=0x7fcbb400a300 comp rx=0 tx=0).stop 2024-05-03T18:45:57.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:56.499+0000 7fcbc75f9700 1 -- 172.21.15.32:0/3979193053 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcbc0103540 msgr2=0x7fcbc00757a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:57.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:56.499+0000 7fcbc75f9700 1 --2- 172.21.15.32:0/3979193053 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcbc0103540 0x7fcbc00757a0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fcbb800b580 tx=0x7fcbb80047b0 comp rx=0 tx=0).stop 2024-05-03T18:45:57.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:56.499+0000 7fcbc5b96700 1 -- 172.21.15.32:0/3979193053 reap_dead start 2024-05-03T18:45:57.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:56.499+0000 7fcbc75f9700 1 -- 172.21.15.32:0/3979193053 shutdown_connections 2024-05-03T18:45:57.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:56.499+0000 7fcbc75f9700 1 -- 172.21.15.32:0/3979193053 >> 172.21.15.32:0/3979193053 conn(0x7fcbc007a650 msgr2=0x7fcbc0102290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:45:57.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:56.499+0000 7fcbc75f9700 1 -- 172.21.15.32:0/3979193053 shutdown_connections 2024-05-03T18:45:57.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:56.499+0000 7fcbc75f9700 1 -- 172.21.15.32:0/3979193053 wait complete. 2024-05-03T18:45:57.300 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:45:57 smithi045 ceph-mon[33299]: pgmap v315: 97 pgs: 97 active+clean; 631 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s; 12 B/s, 0 objects/s recovering 2024-05-03T18:45:57.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:45:57 smithi113 ceph-mon[33540]: pgmap v315: 97 pgs: 97 active+clean; 631 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s; 12 B/s, 0 objects/s recovering 2024-05-03T18:45:57.788 INFO:teuthology.orchestra.run.smithi032.stdout:180388626484 2024-05-03T18:45:57.788 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid f500455e-097b-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-05-03T18:45:57.849 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:45:57.852 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:45:58.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:45:58 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:45:58.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:45:58 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:45:58.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:45:58 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:45:58.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:45:58 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:45:58.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:45:58 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:45:58.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:45:58 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:45:58.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:45:58 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:45:58.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:45:58 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:45:58.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:45:58 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:45:58.304 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:45:58 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:45:58.380 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:45:58 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:45:58.381 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:45:58 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:45:58.381 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:45:58 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:45:58.381 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:45:58 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:45:58.381 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:45:58 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:45:58.482 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:45:58.484 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:45:58.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:58.519+0000 7f3b6dac6700 1 -- 172.21.15.32:0/1306887896 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3b68108400 msgr2=0x7f3b681087e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:58.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:58.519+0000 7f3b6dac6700 1 --2- 172.21.15.32:0/1306887896 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3b68108400 0x7f3b681087e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f3b58009990 tx=0x7f3b58009ca0 comp rx=0 tx=0).stop 2024-05-03T18:45:58.522 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:58.520+0000 7f3b6dac6700 1 -- 172.21.15.32:0/1306887896 shutdown_connections 2024-05-03T18:45:58.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:58.520+0000 7f3b6dac6700 1 --2- 172.21.15.32:0/1306887896 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3b68102d50 0x7f3b6810f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:58.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:58.520+0000 7f3b6dac6700 1 --2- 172.21.15.32:0/1306887896 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3b68108400 0x7f3b681087e0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:58.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:58.520+0000 7f3b6dac6700 1 --2- 172.21.15.32:0/1306887896 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3b68102390 0x7f3b68102810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:58.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:58.520+0000 7f3b6dac6700 1 -- 172.21.15.32:0/1306887896 >> 172.21.15.32:0/1306887896 conn(0x7f3b68079310 msgr2=0x7f3b68079720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:45:58.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:58.520+0000 7f3b6dac6700 1 -- 172.21.15.32:0/1306887896 shutdown_connections 2024-05-03T18:45:58.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:58.520+0000 7f3b6dac6700 1 -- 172.21.15.32:0/1306887896 wait complete. 2024-05-03T18:45:58.523 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:58.521+0000 7f3b6dac6700 1 Processor -- start 2024-05-03T18:45:58.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:58.521+0000 7f3b6dac6700 1 -- start start 2024-05-03T18:45:58.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:58.522+0000 7f3b6dac6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3b68102390 0x7f3b6819f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:58.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:58.522+0000 7f3b6dac6700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3b68102d50 0x7f3b6819fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:58.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:58.522+0000 7f3b6dac6700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3b68108400 0x7f3b68199930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:58.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:58.522+0000 7f3b6dac6700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f3b68113650 con 0x7f3b68102390 2024-05-03T18:45:58.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:58.522+0000 7f3b6dac6700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3b681134d0 con 0x7f3b68108400 2024-05-03T18:45:58.524 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:58.522+0000 7f3b6dac6700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f3b681137d0 con 0x7f3b68102d50 2024-05-03T18:45:58.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:58.522+0000 7f3b677fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3b68102390 0x7f3b6819f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:58.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:58.522+0000 7f3b677fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3b68102390 0x7f3b6819f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53178/0 (socket says 172.21.15.32:53178) 2024-05-03T18:45:58.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:58.522+0000 7f3b67fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3b68108400 0x7f3b68199930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:58.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:58.522+0000 7f3b66ffd700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3b68102d50 0x7f3b6819fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:58.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:58.522+0000 7f3b67fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3b68108400 0x7f3b68199930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.32:36684/0 (socket says 172.21.15.32:36684) 2024-05-03T18:45:58.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:58.522+0000 7f3b66ffd700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3b68102d50 0x7f3b6819fcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.32:46722/0 (socket says 172.21.15.32:46722) 2024-05-03T18:45:58.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:58.522+0000 7f3b677fe700 1 -- 172.21.15.32:0/2472601010 learned_addr learned my addr 172.21.15.32:0/2472601010 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:45:58.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:58.522+0000 7f3b67fff700 1 -- 172.21.15.32:0/2472601010 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3b68102d50 msgr2=0x7f3b6819fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:58.525 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:58.522+0000 7f3b67fff700 1 --2- 172.21.15.32:0/2472601010 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3b68102d50 0x7f3b6819fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:58.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:58.522+0000 7f3b67fff700 1 -- 172.21.15.32:0/2472601010 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3b68102390 msgr2=0x7f3b6819f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:58.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:58.522+0000 7f3b67fff700 1 --2- 172.21.15.32:0/2472601010 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3b68102390 0x7f3b6819f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:58.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:58.522+0000 7f3b67fff700 1 -- 172.21.15.32:0/2472601010 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b6819a090 con 0x7f3b68108400 2024-05-03T18:45:58.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:58.523+0000 7f3b67fff700 1 --2- 172.21.15.32:0/2472601010 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3b68108400 0x7f3b68199930 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f3b5c00e970 tx=0x7f3b5c00ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:45:58.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:58.523+0000 7f3b64ff9700 1 -- 172.21.15.32:0/2472601010 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b5c004220 con 0x7f3b68108400 2024-05-03T18:45:58.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:58.523+0000 7f3b6dac6700 1 -- 172.21.15.32:0/2472601010 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b6819a380 con 0x7f3b68108400 2024-05-03T18:45:58.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:58.523+0000 7f3b64ff9700 1 -- 172.21.15.32:0/2472601010 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b5c0043c0 con 0x7f3b68108400 2024-05-03T18:45:58.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:58.523+0000 7f3b64ff9700 1 -- 172.21.15.32:0/2472601010 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b5c010700 con 0x7f3b68108400 2024-05-03T18:45:58.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:58.523+0000 7f3b6dac6700 1 -- 172.21.15.32:0/2472601010 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b681a6640 con 0x7f3b68108400 2024-05-03T18:45:58.526 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:58.524+0000 7f3b6dac6700 1 -- 172.21.15.32:0/2472601010 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3b68063c30 con 0x7f3b68108400 2024-05-03T18:45:58.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:58.525+0000 7f3b64ff9700 1 -- 172.21.15.32:0/2472601010 <== mon.2 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f3b5c004750 con 0x7f3b68108400 2024-05-03T18:45:58.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:58.526+0000 7f3b64ff9700 1 --2- 172.21.15.32:0/2472601010 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f3b500b2990 0x7f3b500b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:58.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:58.526+0000 7f3b677fe700 1 --2- 172.21.15.32:0/2472601010 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f3b500b2990 0x7f3b500b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:58.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:58.526+0000 7f3b64ff9700 1 -- 172.21.15.32:0/2472601010 <== mon.2 v2:172.21.15.45:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8269+0+0 (secure 0 0 0) 0x7f3b5c0e2310 con 0x7f3b68108400 2024-05-03T18:45:58.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:58.526+0000 7f3b64ff9700 1 --2- 172.21.15.32:0/2472601010 >> [v2:172.21.15.113:6824/703464306,v1:172.21.15.113:6825/703464306] conn(0x7f3b500bc5d0 0x7f3b500bea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:45:58.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:58.526+0000 7f3b64ff9700 1 -- 172.21.15.32:0/2472601010 --> [v2:172.21.15.113:6824/703464306,v1:172.21.15.113:6825/703464306] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3b5c019e50 con 0x7f3b500bc5d0 2024-05-03T18:45:58.528 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:58.526+0000 7f3b64ff9700 1 -- 172.21.15.32:0/2472601010 <== mon.2 v2:172.21.15.45:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7f3b5c0e2700 con 0x7f3b68108400 2024-05-03T18:45:58.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:58.527+0000 7f3b677fe700 1 --2- 172.21.15.32:0/2472601010 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f3b500b2990 0x7f3b500b4e50 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f3b6819aa70 tx=0x7f3b5800b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:45:58.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:58.527+0000 7f3b66ffd700 1 --2- 172.21.15.32:0/2472601010 >> [v2:172.21.15.113:6824/703464306,v1:172.21.15.113:6825/703464306] conn(0x7f3b500bc5d0 0x7f3b500bea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:45:58.529 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:58.527+0000 7f3b66ffd700 1 --2- 172.21.15.32:0/2472601010 >> [v2:172.21.15.113:6824/703464306,v1:172.21.15.113:6825/703464306] conn(0x7f3b500bc5d0 0x7f3b500bea70 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:45:58.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:58.528+0000 7f3b64ff9700 1 -- 172.21.15.32:0/2472601010 <== osd.11 v2:172.21.15.113:6824/703464306 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3b5c019e50 con 0x7f3b500bc5d0 2024-05-03T18:45:58.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:58.545+0000 7f3b6dac6700 1 -- 172.21.15.32:0/2472601010 --> [v2:172.21.15.113:6824/703464306,v1:172.21.15.113:6825/703464306] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3b68064030 con 0x7f3b500bc5d0 2024-05-03T18:45:58.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:58.546+0000 7f3b64ff9700 1 -- 172.21.15.32:0/2472601010 <== osd.11 v2:172.21.15.113:6824/703464306 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f3b68064030 con 0x7f3b500bc5d0 2024-05-03T18:45:58.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:58.546+0000 7f3b6dac6700 1 -- 172.21.15.32:0/2472601010 >> [v2:172.21.15.113:6824/703464306,v1:172.21.15.113:6825/703464306] conn(0x7f3b500bc5d0 msgr2=0x7f3b500bea70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:58.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:58.546+0000 7f3b6dac6700 1 --2- 172.21.15.32:0/2472601010 >> [v2:172.21.15.113:6824/703464306,v1:172.21.15.113:6825/703464306] conn(0x7f3b500bc5d0 0x7f3b500bea70 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:45:58.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:58.546+0000 7f3b6dac6700 1 -- 172.21.15.32:0/2472601010 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f3b500b2990 msgr2=0x7f3b500b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:58.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:58.546+0000 7f3b6dac6700 1 --2- 172.21.15.32:0/2472601010 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f3b500b2990 0x7f3b500b4e50 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f3b6819aa70 tx=0x7f3b5800b260 comp rx=0 tx=0).stop 2024-05-03T18:45:58.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:58.546+0000 7f3b6dac6700 1 -- 172.21.15.32:0/2472601010 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3b68108400 msgr2=0x7f3b68199930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:45:58.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:58.546+0000 7f3b6dac6700 1 --2- 172.21.15.32:0/2472601010 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3b68108400 0x7f3b68199930 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f3b5c00e970 tx=0x7f3b5c00ec80 comp rx=0 tx=0).stop 2024-05-03T18:45:58.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:58.546+0000 7f3b67fff700 1 -- 172.21.15.32:0/2472601010 reap_dead start 2024-05-03T18:45:58.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:58.546+0000 7f3b6dac6700 1 -- 172.21.15.32:0/2472601010 shutdown_connections 2024-05-03T18:45:58.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:58.546+0000 7f3b6dac6700 1 -- 172.21.15.32:0/2472601010 >> 172.21.15.32:0/2472601010 conn(0x7f3b68079310 msgr2=0x7f3b68101090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:45:58.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:58.546+0000 7f3b6dac6700 1 -- 172.21.15.32:0/2472601010 shutdown_connections 2024-05-03T18:45:58.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:45:58.546+0000 7f3b6dac6700 1 -- 172.21.15.32:0/2472601010 wait complete. 2024-05-03T18:45:59.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:45:59 smithi032 ceph-mon[29763]: pgmap v316: 97 pgs: 97 active+clean; 631 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s; 11 B/s, 0 objects/s recovering 2024-05-03T18:45:59.355 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:45:59 smithi045 ceph-mon[33299]: pgmap v316: 97 pgs: 97 active+clean; 631 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s; 11 B/s, 0 objects/s recovering 2024-05-03T18:45:59.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:45:59 smithi113 ceph-mon[33540]: pgmap v316: 97 pgs: 97 active+clean; 631 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s; 11 B/s, 0 objects/s recovering 2024-05-03T18:46:00.902 INFO:teuthology.orchestra.run.smithi032.stdout:300647710748 2024-05-03T18:46:00.903 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid f500455e-097b-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-05-03T18:46:01.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:46:01 smithi032 ceph-mon[29763]: pgmap v317: 97 pgs: 97 active+clean; 631 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s; 9 B/s, 0 objects/s recovering 2024-05-03T18:46:01.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:46:01 smithi045 ceph-mon[33299]: pgmap v317: 97 pgs: 97 active+clean; 631 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s; 9 B/s, 0 objects/s recovering 2024-05-03T18:46:01.553 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:46:01 smithi113 ceph-mon[33540]: pgmap v317: 97 pgs: 97 active+clean; 631 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s; 9 B/s, 0 objects/s recovering 2024-05-03T18:46:02.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:02.682+0000 7f7ed5d22700 1 -- 172.21.15.32:0/2729764399 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ed0109510 msgr2=0x7f7ed01098f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:02.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:02.682+0000 7f7ed5d22700 1 --2- 172.21.15.32:0/2729764399 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ed0109510 0x7f7ed01098f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f7ec0009990 tx=0x7f7ec0009ca0 comp rx=0 tx=0).stop 2024-05-03T18:46:02.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:02.682+0000 7f7ed5d22700 1 -- 172.21.15.32:0/2729764399 shutdown_connections 2024-05-03T18:46:02.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:02.682+0000 7f7ed5d22700 1 --2- 172.21.15.32:0/2729764399 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7ed006bb40 0x7f7ed0110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:02.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:02.682+0000 7f7ed5d22700 1 --2- 172.21.15.32:0/2729764399 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ed0109510 0x7f7ed01098f0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:02.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:02.682+0000 7f7ed5d22700 1 --2- 172.21.15.32:0/2729764399 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ed006b180 0x7f7ed006b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:02.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:02.682+0000 7f7ed5d22700 1 -- 172.21.15.32:0/2729764399 >> 172.21.15.32:0/2729764399 conn(0x7f7ed0079350 msgr2=0x7f7ed0079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:46:02.684 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:02.682+0000 7f7ed5d22700 1 -- 172.21.15.32:0/2729764399 shutdown_connections 2024-05-03T18:46:02.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:02.682+0000 7f7ed5d22700 1 -- 172.21.15.32:0/2729764399 wait complete. 2024-05-03T18:46:02.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:02.683+0000 7f7ed5d22700 1 Processor -- start 2024-05-03T18:46:02.685 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:02.684+0000 7f7ed5d22700 1 -- start start 2024-05-03T18:46:02.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:02.684+0000 7f7ed5d22700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ed006b180 0x7f7ed010ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:02.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:02.684+0000 7f7ed5d22700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7ed006bb40 0x7f7ed010d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:02.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:02.684+0000 7f7ed5d22700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ed0109510 0x7f7ed010d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:02.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:02.684+0000 7f7ed5d22700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f7ed0103c10 con 0x7f7ed0109510 2024-05-03T18:46:02.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:02.684+0000 7f7ed5d22700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7ed0103a90 con 0x7f7ed006b180 2024-05-03T18:46:02.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:02.684+0000 7f7ed5d22700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f7ed0103d90 con 0x7f7ed006bb40 2024-05-03T18:46:02.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:02.685+0000 7f7ecffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ed0109510 0x7f7ed010d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:02.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:02.685+0000 7f7ecffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ed0109510 0x7f7ed010d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53194/0 (socket says 172.21.15.32:53194) 2024-05-03T18:46:02.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:02.685+0000 7f7ecffff700 1 -- 172.21.15.32:0/4200714827 learned_addr learned my addr 172.21.15.32:0/4200714827 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:46:02.686 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:02.685+0000 7f7ecf7fe700 1 --2- 172.21.15.32:0/4200714827 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ed006b180 0x7f7ed010ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:02.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:02.685+0000 7f7ecffff700 1 -- 172.21.15.32:0/4200714827 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7ed006bb40 msgr2=0x7f7ed010d060 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-03T18:46:02.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:02.685+0000 7f7ecffff700 1 --2- 172.21.15.32:0/4200714827 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7ed006bb40 0x7f7ed010d060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:02.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:02.685+0000 7f7ecffff700 1 -- 172.21.15.32:0/4200714827 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ed006b180 msgr2=0x7f7ed010ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:02.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:02.685+0000 7f7ecffff700 1 --2- 172.21.15.32:0/4200714827 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ed006b180 0x7f7ed010ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:02.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:02.685+0000 7f7ecffff700 1 -- 172.21.15.32:0/4200714827 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ed010dca0 con 0x7f7ed0109510 2024-05-03T18:46:02.687 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:02.686+0000 7f7ecffff700 1 --2- 172.21.15.32:0/4200714827 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ed0109510 0x7f7ed010d5a0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f7ec8007d70 tx=0x7f7ec800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:46:02.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:02.686+0000 7f7eccff9700 1 -- 172.21.15.32:0/4200714827 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ec8016020 con 0x7f7ed0109510 2024-05-03T18:46:02.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:02.686+0000 7f7eccff9700 1 -- 172.21.15.32:0/4200714827 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ec800ad40 con 0x7f7ed0109510 2024-05-03T18:46:02.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:02.686+0000 7f7eccff9700 1 -- 172.21.15.32:0/4200714827 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ec80115e0 con 0x7f7ed0109510 2024-05-03T18:46:02.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:02.686+0000 7f7ed5d22700 1 -- 172.21.15.32:0/4200714827 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ed01ac740 con 0x7f7ed0109510 2024-05-03T18:46:02.688 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:02.686+0000 7f7ed5d22700 1 -- 172.21.15.32:0/4200714827 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ed01acba0 con 0x7f7ed0109510 2024-05-03T18:46:02.689 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:02.687+0000 7f7ed5d22700 1 -- 172.21.15.32:0/4200714827 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7ed01016d0 con 0x7f7ed0109510 2024-05-03T18:46:02.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:02.691+0000 7f7eccff9700 1 -- 172.21.15.32:0/4200714827 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f7ec8019020 con 0x7f7ed0109510 2024-05-03T18:46:02.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:02.692+0000 7f7eccff9700 1 --2- 172.21.15.32:0/4200714827 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f7ebc0b2a60 0x7f7ebc0b4f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:02.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:02.692+0000 7f7ecf7fe700 1 --2- 172.21.15.32:0/4200714827 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f7ebc0b2a60 0x7f7ebc0b4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:02.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:02.692+0000 7f7eccff9700 1 -- 172.21.15.32:0/4200714827 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8269+0+0 (secure 0 0 0) 0x7f7ec80e26d0 con 0x7f7ed0109510 2024-05-03T18:46:02.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:02.692+0000 7f7eccff9700 1 --2- 172.21.15.32:0/4200714827 >> [v2:172.21.15.45:6808/71932118,v1:172.21.15.45:6809/71932118] conn(0x7f7ebc0bc6a0 0x7f7ebc0beb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:02.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:02.692+0000 7f7eccff9700 1 -- 172.21.15.32:0/4200714827 --> [v2:172.21.15.45:6808/71932118,v1:172.21.15.45:6809/71932118] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7ebc0bf210 con 0x7f7ebc0bc6a0 2024-05-03T18:46:02.694 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:02.692+0000 7f7eccff9700 1 -- 172.21.15.32:0/4200714827 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7f7ec80e2b20 con 0x7f7ed0109510 2024-05-03T18:46:02.694 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:46:02.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:02.693+0000 7f7eceffd700 1 --2- 172.21.15.32:0/4200714827 >> [v2:172.21.15.45:6808/71932118,v1:172.21.15.45:6809/71932118] conn(0x7f7ebc0bc6a0 0x7f7ebc0beb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:02.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:02.693+0000 7f7ecf7fe700 1 --2- 172.21.15.32:0/4200714827 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f7ebc0b2a60 0x7f7ebc0b4f20 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f7ec00098e0 tx=0x7f7ec0005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:46:02.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:02.693+0000 7f7eceffd700 1 --2- 172.21.15.32:0/4200714827 >> [v2:172.21.15.45:6808/71932118,v1:172.21.15.45:6809/71932118] conn(0x7f7ebc0bc6a0 0x7f7ebc0beb40 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:46:02.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:02.694+0000 7f7eccff9700 1 -- 172.21.15.32:0/4200714827 <== osd.4 v2:172.21.15.45:6808/71932118 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7ebc0bf210 con 0x7f7ebc0bc6a0 2024-05-03T18:46:02.696 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:46:02.726 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:02.724+0000 7f7ed5d22700 1 -- 172.21.15.32:0/4200714827 --> [v2:172.21.15.45:6808/71932118,v1:172.21.15.45:6809/71932118] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7ed0104bd0 con 0x7f7ebc0bc6a0 2024-05-03T18:46:02.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:02.726+0000 7f7eccff9700 1 -- 172.21.15.32:0/4200714827 <== osd.4 v2:172.21.15.45:6808/71932118 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f7ed0104bd0 con 0x7f7ebc0bc6a0 2024-05-03T18:46:02.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:02.726+0000 7f7ed5d22700 1 -- 172.21.15.32:0/4200714827 >> [v2:172.21.15.45:6808/71932118,v1:172.21.15.45:6809/71932118] conn(0x7f7ebc0bc6a0 msgr2=0x7f7ebc0beb40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:02.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:02.726+0000 7f7ed5d22700 1 --2- 172.21.15.32:0/4200714827 >> [v2:172.21.15.45:6808/71932118,v1:172.21.15.45:6809/71932118] conn(0x7f7ebc0bc6a0 0x7f7ebc0beb40 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:02.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:02.726+0000 7f7ed5d22700 1 -- 172.21.15.32:0/4200714827 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f7ebc0b2a60 msgr2=0x7f7ebc0b4f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:02.728 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:02.726+0000 7f7ed5d22700 1 --2- 172.21.15.32:0/4200714827 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f7ebc0b2a60 0x7f7ebc0b4f20 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f7ec00098e0 tx=0x7f7ec0005960 comp rx=0 tx=0).stop 2024-05-03T18:46:02.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:02.726+0000 7f7ed5d22700 1 -- 172.21.15.32:0/4200714827 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ed0109510 msgr2=0x7f7ed010d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:02.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:02.726+0000 7f7ed5d22700 1 --2- 172.21.15.32:0/4200714827 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ed0109510 0x7f7ed010d5a0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f7ec8007d70 tx=0x7f7ec800a340 comp rx=0 tx=0).stop 2024-05-03T18:46:02.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:02.726+0000 7f7ecffff700 1 -- 172.21.15.32:0/4200714827 reap_dead start 2024-05-03T18:46:02.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:02.726+0000 7f7ed5d22700 1 -- 172.21.15.32:0/4200714827 shutdown_connections 2024-05-03T18:46:02.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:02.726+0000 7f7ed5d22700 1 -- 172.21.15.32:0/4200714827 >> 172.21.15.32:0/4200714827 conn(0x7f7ed0079350 msgr2=0x7f7ed0100c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:46:02.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:02.727+0000 7f7ed5d22700 1 -- 172.21.15.32:0/4200714827 shutdown_connections 2024-05-03T18:46:02.729 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:02.727+0000 7f7ed5d22700 1 -- 172.21.15.32:0/4200714827 wait complete. 2024-05-03T18:46:03.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:46:03 smithi032 ceph-mon[29763]: pgmap v318: 97 pgs: 97 active+clean; 631 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 9 B/s, 0 objects/s recovering 2024-05-03T18:46:03.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:46:03 smithi045 ceph-mon[33299]: pgmap v318: 97 pgs: 97 active+clean; 631 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 9 B/s, 0 objects/s recovering 2024-05-03T18:46:03.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:46:03 smithi113 ceph-mon[33540]: pgmap v318: 97 pgs: 97 active+clean; 631 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 9 B/s, 0 objects/s recovering 2024-05-03T18:46:04.312 INFO:teuthology.orchestra.run.smithi032.stdout:133143986240 2024-05-03T18:46:04.312 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid f500455e-097b-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-05-03T18:46:04.438 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:46:04.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:04.551+0000 7f960e75a700 1 -- 172.21.15.32:0/3932545405 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9608107760 msgr2=0x7f960810f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:04.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:04.551+0000 7f960e75a700 1 --2- 172.21.15.32:0/3932545405 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9608107760 0x7f960810f2e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f960000b0d0 tx=0x7f960000b3e0 comp rx=0 tx=0).stop 2024-05-03T18:46:04.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:04.551+0000 7f960e75a700 1 -- 172.21.15.32:0/3932545405 shutdown_connections 2024-05-03T18:46:04.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:04.551+0000 7f960e75a700 1 --2- 172.21.15.32:0/3932545405 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9608107760 0x7f960810f2e0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:04.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:04.551+0000 7f960e75a700 1 --2- 172.21.15.32:0/3932545405 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9608102e00 0x7f96081031e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:04.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:04.551+0000 7f960e75a700 1 --2- 172.21.15.32:0/3932545405 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f96081037b0 0x7f9608107030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:04.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:04.551+0000 7f960e75a700 1 -- 172.21.15.32:0/3932545405 >> 172.21.15.32:0/3932545405 conn(0x7f9608078a10 msgr2=0x7f96080ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:46:04.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:04.551+0000 7f960e75a700 1 -- 172.21.15.32:0/3932545405 shutdown_connections 2024-05-03T18:46:04.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:04.551+0000 7f960e75a700 1 -- 172.21.15.32:0/3932545405 wait complete. 2024-05-03T18:46:04.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:04.552+0000 7f960e75a700 1 Processor -- start 2024-05-03T18:46:04.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:04.552+0000 7f960e75a700 1 -- start start 2024-05-03T18:46:04.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:04.552+0000 7f960e75a700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9608102e00 0x7f9608109380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:04.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:04.552+0000 7f960e75a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f96081037b0 0x7f96081098c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:04.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:04.552+0000 7f960e75a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9608107760 0x7f9608109e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:04.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:04.552+0000 7f960e75a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f9608112550 con 0x7f9608107760 2024-05-03T18:46:04.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:04.552+0000 7f960e75a700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f96081123d0 con 0x7f96081037b0 2024-05-03T18:46:04.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:04.552+0000 7f960e75a700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f96081126d0 con 0x7f9608102e00 2024-05-03T18:46:04.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:04.553+0000 7f960ccf7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9608107760 0x7f9608109e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:04.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:04.553+0000 7f960ccf7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9608107760 0x7f9608109e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55298/0 (socket says 172.21.15.32:55298) 2024-05-03T18:46:04.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:04.553+0000 7f960ccf7700 1 -- 172.21.15.32:0/2605215327 learned_addr learned my addr 172.21.15.32:0/2605215327 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:46:04.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:04.553+0000 7f9607fff700 1 --2- 172.21.15.32:0/2605215327 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9608102e00 0x7f9608109380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:04.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:04.553+0000 7f960ccf7700 1 -- 172.21.15.32:0/2605215327 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9608102e00 msgr2=0x7f9608109380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:04.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:04.553+0000 7f960ccf7700 1 --2- 172.21.15.32:0/2605215327 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9608102e00 0x7f9608109380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:04.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:04.553+0000 7f96077fe700 1 --2- 172.21.15.32:0/2605215327 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f96081037b0 0x7f96081098c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:04.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:04.553+0000 7f960ccf7700 1 -- 172.21.15.32:0/2605215327 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f96081037b0 msgr2=0x7f96081098c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:04.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:04.553+0000 7f960ccf7700 1 --2- 172.21.15.32:0/2605215327 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f96081037b0 0x7f96081098c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:04.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:04.553+0000 7f960ccf7700 1 -- 172.21.15.32:0/2605215327 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96081a8060 con 0x7f9608107760 2024-05-03T18:46:04.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:04.553+0000 7f96077fe700 1 --2- 172.21.15.32:0/2605215327 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f96081037b0 0x7f96081098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:46:04.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:04.553+0000 7f960ccf7700 1 --2- 172.21.15.32:0/2605215327 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9608107760 0x7f9608109e00 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f9600009fd0 tx=0x7f9600007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:46:04.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:04.553+0000 7f96057fa700 1 -- 172.21.15.32:0/2605215327 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f960001f070 con 0x7f9608107760 2024-05-03T18:46:04.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:04.553+0000 7f96057fa700 1 -- 172.21.15.32:0/2605215327 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9600003c00 con 0x7f9608107760 2024-05-03T18:46:04.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:04.553+0000 7f960e75a700 1 -- 172.21.15.32:0/2605215327 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96081a82f0 con 0x7f9608107760 2024-05-03T18:46:04.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:04.553+0000 7f96057fa700 1 -- 172.21.15.32:0/2605215327 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f960001b7b0 con 0x7f9608107760 2024-05-03T18:46:04.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:04.553+0000 7f960e75a700 1 -- 172.21.15.32:0/2605215327 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96081a87a0 con 0x7f9608107760 2024-05-03T18:46:04.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:04.554+0000 7f960e75a700 1 -- 172.21.15.32:0/2605215327 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f96081060f0 con 0x7f9608107760 2024-05-03T18:46:04.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:04.556+0000 7f96057fa700 1 -- 172.21.15.32:0/2605215327 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f9600022070 con 0x7f9608107760 2024-05-03T18:46:04.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:04.556+0000 7f96057fa700 1 --2- 172.21.15.32:0/2605215327 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f95f00b2990 0x7f95f00b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:04.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:04.557+0000 7f9607fff700 1 --2- 172.21.15.32:0/2605215327 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f95f00b2990 0x7f95f00b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:04.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:04.557+0000 7f96057fa700 1 -- 172.21.15.32:0/2605215327 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8269+0+0 (secure 0 0 0) 0x7f960000f020 con 0x7f9608107760 2024-05-03T18:46:04.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:04.557+0000 7f96057fa700 1 --2- 172.21.15.32:0/2605215327 >> [v2:172.21.15.113:6816/3569181558,v1:172.21.15.113:6817/3569181558] conn(0x7f95f00bc5d0 0x7f95f00bea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:04.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:04.557+0000 7f96057fa700 1 -- 172.21.15.32:0/2605215327 --> [v2:172.21.15.113:6816/3569181558,v1:172.21.15.113:6817/3569181558] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f95f00bf140 con 0x7f95f00bc5d0 2024-05-03T18:46:04.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:04.557+0000 7f96057fa700 1 -- 172.21.15.32:0/2605215327 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7f96000e3c00 con 0x7f9608107760 2024-05-03T18:46:04.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:04.557+0000 7f96077fe700 1 --2- 172.21.15.32:0/2605215327 >> [v2:172.21.15.113:6816/3569181558,v1:172.21.15.113:6817/3569181558] conn(0x7f95f00bc5d0 0x7f95f00bea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:04.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:04.557+0000 7f9607fff700 1 --2- 172.21.15.32:0/2605215327 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f95f00b2990 0x7f95f00b4e50 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f95f8005950 tx=0x7f95f800a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:46:04.559 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:04.557+0000 7f96077fe700 1 --2- 172.21.15.32:0/2605215327 >> [v2:172.21.15.113:6816/3569181558,v1:172.21.15.113:6817/3569181558] conn(0x7f95f00bc5d0 0x7f95f00bea70 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:46:04.560 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:04.558+0000 7f96057fa700 1 -- 172.21.15.32:0/2605215327 <== osd.8 v2:172.21.15.113:6816/3569181558 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f95f00bf140 con 0x7f95f00bc5d0 2024-05-03T18:46:04.576 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:04.575+0000 7f960e75a700 1 -- 172.21.15.32:0/2605215327 --> [v2:172.21.15.113:6816/3569181558,v1:172.21.15.113:6817/3569181558] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9608103bb0 con 0x7f95f00bc5d0 2024-05-03T18:46:04.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:04.576+0000 7f96057fa700 1 -- 172.21.15.32:0/2605215327 <== osd.8 v2:172.21.15.113:6816/3569181558 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f9608103bb0 con 0x7f95f00bc5d0 2024-05-03T18:46:04.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:04.576+0000 7f960e75a700 1 -- 172.21.15.32:0/2605215327 >> [v2:172.21.15.113:6816/3569181558,v1:172.21.15.113:6817/3569181558] conn(0x7f95f00bc5d0 msgr2=0x7f95f00bea70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:04.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:04.576+0000 7f960e75a700 1 --2- 172.21.15.32:0/2605215327 >> [v2:172.21.15.113:6816/3569181558,v1:172.21.15.113:6817/3569181558] conn(0x7f95f00bc5d0 0x7f95f00bea70 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:04.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:04.576+0000 7f960e75a700 1 -- 172.21.15.32:0/2605215327 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f95f00b2990 msgr2=0x7f95f00b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:04.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:04.576+0000 7f960e75a700 1 --2- 172.21.15.32:0/2605215327 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f95f00b2990 0x7f95f00b4e50 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f95f8005950 tx=0x7f95f800a300 comp rx=0 tx=0).stop 2024-05-03T18:46:04.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:04.576+0000 7f960e75a700 1 -- 172.21.15.32:0/2605215327 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9608107760 msgr2=0x7f9608109e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:04.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:04.576+0000 7f960e75a700 1 --2- 172.21.15.32:0/2605215327 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9608107760 0x7f9608109e00 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f9600009fd0 tx=0x7f9600007a20 comp rx=0 tx=0).stop 2024-05-03T18:46:04.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:04.576+0000 7f960ccf7700 1 -- 172.21.15.32:0/2605215327 reap_dead start 2024-05-03T18:46:04.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:04.577+0000 7f960e75a700 1 -- 172.21.15.32:0/2605215327 shutdown_connections 2024-05-03T18:46:04.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:04.577+0000 7f960e75a700 1 -- 172.21.15.32:0/2605215327 >> 172.21.15.32:0/2605215327 conn(0x7f9608078a10 msgr2=0x7f9608104e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:46:04.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:04.577+0000 7f960e75a700 1 -- 172.21.15.32:0/2605215327 shutdown_connections 2024-05-03T18:46:04.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:04.577+0000 7f960e75a700 1 -- 172.21.15.32:0/2605215327 wait complete. 2024-05-03T18:46:05.048 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:46:05.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:46:05 smithi032 ceph-mon[29763]: pgmap v319: 97 pgs: 97 active+clean; 631 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:46:05.497 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:46:05 smithi045 ceph-mon[33299]: pgmap v319: 97 pgs: 97 active+clean; 631 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:46:05.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:46:05 smithi113 ceph-mon[33540]: pgmap v319: 97 pgs: 97 active+clean; 631 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:46:05.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:05.949+0000 7fb682c8e700 1 -- 172.21.15.32:0/1559011736 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb67c101a90 msgr2=0x7fb67c101f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:05.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:05.949+0000 7fb682c8e700 1 --2- 172.21.15.32:0/1559011736 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb67c101a90 0x7fb67c101f10 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fb66c009990 tx=0x7fb66c009ca0 comp rx=0 tx=0).stop 2024-05-03T18:46:05.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:05.949+0000 7fb682c8e700 1 -- 172.21.15.32:0/1559011736 shutdown_connections 2024-05-03T18:46:05.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:05.949+0000 7fb682c8e700 1 --2- 172.21.15.32:0/1559011736 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb67c102450 0x7fb67c110340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:05.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:05.949+0000 7fb682c8e700 1 --2- 172.21.15.32:0/1559011736 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb67c109520 0x7fb67c109900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:05.950 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:05.949+0000 7fb682c8e700 1 --2- 172.21.15.32:0/1559011736 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb67c101a90 0x7fb67c101f10 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:05.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:05.949+0000 7fb682c8e700 1 -- 172.21.15.32:0/1559011736 >> 172.21.15.32:0/1559011736 conn(0x7fb67c078a10 msgr2=0x7fb67c0ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:46:05.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:05.949+0000 7fb682c8e700 1 -- 172.21.15.32:0/1559011736 shutdown_connections 2024-05-03T18:46:05.951 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:05.949+0000 7fb682c8e700 1 -- 172.21.15.32:0/1559011736 wait complete. 2024-05-03T18:46:05.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:05.950+0000 7fb682c8e700 1 Processor -- start 2024-05-03T18:46:05.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:05.950+0000 7fb682c8e700 1 -- start start 2024-05-03T18:46:05.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:05.950+0000 7fb682c8e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb67c101a90 0x7fb67c19f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:05.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:05.950+0000 7fb682c8e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb67c102450 0x7fb67c19fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:05.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:05.950+0000 7fb682c8e700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb67c109520 0x7fb67c199930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:05.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:05.950+0000 7fb682c8e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fb67c113650 con 0x7fb67c102450 2024-05-03T18:46:05.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:05.950+0000 7fb682c8e700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb67c1134d0 con 0x7fb67c101a90 2024-05-03T18:46:05.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:05.950+0000 7fb682c8e700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fb67c1137d0 con 0x7fb67c109520 2024-05-03T18:46:05.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:05.950+0000 7fb67bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb67c102450 0x7fb67c19fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:05.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:05.950+0000 7fb67bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb67c102450 0x7fb67c19fcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55310/0 (socket says 172.21.15.32:55310) 2024-05-03T18:46:05.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:05.950+0000 7fb67bfff700 1 -- 172.21.15.32:0/709770749 learned_addr learned my addr 172.21.15.32:0/709770749 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:46:05.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:05.950+0000 7fb68122b700 1 --2- 172.21.15.32:0/709770749 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb67c109520 0x7fb67c199930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:05.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:05.950+0000 7fb680a2a700 1 --2- 172.21.15.32:0/709770749 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb67c101a90 0x7fb67c19f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:05.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:05.951+0000 7fb68122b700 1 -- 172.21.15.32:0/709770749 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb67c101a90 msgr2=0x7fb67c19f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:05.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:05.951+0000 7fb68122b700 1 --2- 172.21.15.32:0/709770749 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb67c101a90 0x7fb67c19f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:05.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:05.951+0000 7fb68122b700 1 -- 172.21.15.32:0/709770749 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb67c102450 msgr2=0x7fb67c19fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:05.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:05.951+0000 7fb68122b700 1 --2- 172.21.15.32:0/709770749 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb67c102450 0x7fb67c19fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:05.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:05.951+0000 7fb68122b700 1 -- 172.21.15.32:0/709770749 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb67c19a090 con 0x7fb67c109520 2024-05-03T18:46:05.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:05.951+0000 7fb67bfff700 1 --2- 172.21.15.32:0/709770749 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb67c102450 0x7fb67c19fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T18:46:05.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:05.951+0000 7fb68122b700 1 --2- 172.21.15.32:0/709770749 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb67c109520 0x7fb67c199930 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb67400efc0 tx=0x7fb67400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:46:05.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:05.951+0000 7fb679ffb700 1 -- 172.21.15.32:0/709770749 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6740092e0 con 0x7fb67c109520 2024-05-03T18:46:05.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:05.951+0000 7fb679ffb700 1 -- 172.21.15.32:0/709770749 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb674004bb0 con 0x7fb67c109520 2024-05-03T18:46:05.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:05.951+0000 7fb679ffb700 1 -- 172.21.15.32:0/709770749 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb674004020 con 0x7fb67c109520 2024-05-03T18:46:05.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:05.951+0000 7fb682c8e700 1 -- 172.21.15.32:0/709770749 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb67c19a320 con 0x7fb67c109520 2024-05-03T18:46:05.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:05.951+0000 7fb682c8e700 1 -- 172.21.15.32:0/709770749 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb67c1a6640 con 0x7fb67c109520 2024-05-03T18:46:05.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:05.952+0000 7fb682c8e700 1 -- 172.21.15.32:0/709770749 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb67c063c30 con 0x7fb67c109520 2024-05-03T18:46:05.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:05.954+0000 7fb679ffb700 1 -- 172.21.15.32:0/709770749 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fb67401b030 con 0x7fb67c109520 2024-05-03T18:46:05.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:05.954+0000 7fb679ffb700 1 --2- 172.21.15.32:0/709770749 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fb6680b2910 0x7fb6680b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:05.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:05.955+0000 7fb680a2a700 1 --2- 172.21.15.32:0/709770749 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fb6680b2910 0x7fb6680b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:05.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:05.955+0000 7fb679ffb700 1 -- 172.21.15.32:0/709770749 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8269+0+0 (secure 0 0 0) 0x7fb6740e28b0 con 0x7fb67c109520 2024-05-03T18:46:05.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:05.955+0000 7fb679ffb700 1 --2- 172.21.15.32:0/709770749 >> [v2:172.21.15.113:6800/2308128926,v1:172.21.15.113:6801/2308128926] conn(0x7fb6680bc550 0x7fb6680be9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:05.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:05.955+0000 7fb679ffb700 1 -- 172.21.15.32:0/709770749 --> [v2:172.21.15.113:6800/2308128926,v1:172.21.15.113:6801/2308128926] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb674011e50 con 0x7fb6680bc550 2024-05-03T18:46:05.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:05.955+0000 7fb679ffb700 1 -- 172.21.15.32:0/709770749 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7fb6740e2c60 con 0x7fb67c109520 2024-05-03T18:46:05.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:05.955+0000 7fb680a2a700 1 --2- 172.21.15.32:0/709770749 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fb6680b2910 0x7fb6680b4dd0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fb67c19aa70 tx=0x7fb664009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:46:05.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:05.955+0000 7fb67bfff700 1 --2- 172.21.15.32:0/709770749 >> [v2:172.21.15.113:6800/2308128926,v1:172.21.15.113:6801/2308128926] conn(0x7fb6680bc550 0x7fb6680be9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:05.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:05.956+0000 7fb67bfff700 1 --2- 172.21.15.32:0/709770749 >> [v2:172.21.15.113:6800/2308128926,v1:172.21.15.113:6801/2308128926] conn(0x7fb6680bc550 0x7fb6680be9f0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:46:05.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:05.956+0000 7fb679ffb700 1 -- 172.21.15.32:0/709770749 <== osd.2 v2:172.21.15.113:6800/2308128926 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb674011e50 con 0x7fb6680bc550 2024-05-03T18:46:05.974 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:05.974+0000 7fb682c8e700 1 -- 172.21.15.32:0/709770749 --> [v2:172.21.15.113:6800/2308128926,v1:172.21.15.113:6801/2308128926] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb67c0640b0 con 0x7fb6680bc550 2024-05-03T18:46:05.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:05.974+0000 7fb679ffb700 1 -- 172.21.15.32:0/709770749 <== osd.2 v2:172.21.15.113:6800/2308128926 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fb67c0640b0 con 0x7fb6680bc550 2024-05-03T18:46:05.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:05.974+0000 7fb682c8e700 1 -- 172.21.15.32:0/709770749 >> [v2:172.21.15.113:6800/2308128926,v1:172.21.15.113:6801/2308128926] conn(0x7fb6680bc550 msgr2=0x7fb6680be9f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:05.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:05.974+0000 7fb682c8e700 1 --2- 172.21.15.32:0/709770749 >> [v2:172.21.15.113:6800/2308128926,v1:172.21.15.113:6801/2308128926] conn(0x7fb6680bc550 0x7fb6680be9f0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:05.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:05.975+0000 7fb682c8e700 1 -- 172.21.15.32:0/709770749 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fb6680b2910 msgr2=0x7fb6680b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:05.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:05.975+0000 7fb682c8e700 1 --2- 172.21.15.32:0/709770749 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fb6680b2910 0x7fb6680b4dd0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fb67c19aa70 tx=0x7fb664009260 comp rx=0 tx=0).stop 2024-05-03T18:46:05.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:05.975+0000 7fb682c8e700 1 -- 172.21.15.32:0/709770749 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb67c109520 msgr2=0x7fb67c199930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:05.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:05.975+0000 7fb682c8e700 1 --2- 172.21.15.32:0/709770749 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb67c109520 0x7fb67c199930 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb67400efc0 tx=0x7fb67400c3a0 comp rx=0 tx=0).stop 2024-05-03T18:46:05.976 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:05.975+0000 7fb68122b700 1 -- 172.21.15.32:0/709770749 reap_dead start 2024-05-03T18:46:05.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:05.975+0000 7fb682c8e700 1 -- 172.21.15.32:0/709770749 shutdown_connections 2024-05-03T18:46:05.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:05.975+0000 7fb682c8e700 1 -- 172.21.15.32:0/709770749 >> 172.21.15.32:0/709770749 conn(0x7fb67c078a10 msgr2=0x7fb67c1002b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:46:05.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:05.975+0000 7fb682c8e700 1 -- 172.21.15.32:0/709770749 shutdown_connections 2024-05-03T18:46:05.977 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:05.975+0000 7fb682c8e700 1 -- 172.21.15.32:0/709770749 wait complete. 2024-05-03T18:46:06.629 INFO:teuthology.orchestra.run.smithi032.stdout:227633266732 2024-05-03T18:46:06.630 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid f500455e-097b-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-05-03T18:46:07.215 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:46:07 smithi032 ceph-mon[29763]: pgmap v320: 97 pgs: 97 active+clean; 631 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:46:07.497 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:46:07 smithi045 ceph-mon[33299]: pgmap v320: 97 pgs: 97 active+clean; 631 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:46:07.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:46:07 smithi113 ceph-mon[33540]: pgmap v320: 97 pgs: 97 active+clean; 631 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:46:09.248 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:46:09 smithi045 ceph-mon[33299]: pgmap v321: 97 pgs: 97 active+clean; 631 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:46:09.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:46:09 smithi032 ceph-mon[29763]: pgmap v321: 97 pgs: 97 active+clean; 631 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:46:09.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:46:09 smithi113 ceph-mon[33540]: pgmap v321: 97 pgs: 97 active+clean; 631 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:46:10.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:46:10 smithi032 ceph-mon[29763]: from='client.? 172.21.15.32:0/1616850784' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-03T18:46:10.497 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:46:10 smithi045 ceph-mon[33299]: from='client.? 172.21.15.32:0/1616850784' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-03T18:46:10.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:46:10 smithi113 ceph-mon[33540]: from='client.? 172.21.15.32:0/1616850784' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-03T18:46:10.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:09.667+0000 7fd63dd84700 1 -- 172.21.15.32:0/3822980234 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd638103060 msgr2=0x7fd63810f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:10.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:09.667+0000 7fd63dd84700 1 --2- 172.21.15.32:0/3822980234 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd638103060 0x7fd63810f1f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fd63000b0d0 tx=0x7fd63000b3e0 comp rx=0 tx=0).stop 2024-05-03T18:46:10.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:09.667+0000 7fd63dd84700 1 -- 172.21.15.32:0/3822980234 shutdown_connections 2024-05-03T18:46:10.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:09.667+0000 7fd63dd84700 1 --2- 172.21.15.32:0/3822980234 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd638103060 0x7fd63810f1f0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:10.703 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:09.667+0000 7fd63dd84700 1 --2- 172.21.15.32:0/3822980234 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd6381084d0 0x7fd6381088b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:10.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:09.667+0000 7fd63dd84700 1 --2- 172.21.15.32:0/3822980234 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd6381026c0 0x7fd638102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:10.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:09.667+0000 7fd63dd84700 1 -- 172.21.15.32:0/3822980234 >> 172.21.15.32:0/3822980234 conn(0x7fd638079170 msgr2=0x7fd638079580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:46:10.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:09.667+0000 7fd63dd84700 1 -- 172.21.15.32:0/3822980234 shutdown_connections 2024-05-03T18:46:10.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:09.667+0000 7fd63dd84700 1 -- 172.21.15.32:0/3822980234 wait complete. 2024-05-03T18:46:10.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:09.667+0000 7fd63dd84700 1 Processor -- start 2024-05-03T18:46:10.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:09.668+0000 7fd63dd84700 1 -- start start 2024-05-03T18:46:10.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:09.668+0000 7fd63dd84700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd6381026c0 0x7fd638114a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:10.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:09.668+0000 7fd63dd84700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd638103060 0x7fd638112db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:10.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:09.668+0000 7fd63dd84700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd6381084d0 0x7fd638113520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:10.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:09.668+0000 7fd63dd84700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd6381125f0 con 0x7fd6381026c0 2024-05-03T18:46:10.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:09.668+0000 7fd63dd84700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd638112470 con 0x7fd638103060 2024-05-03T18:46:10.704 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:09.668+0000 7fd63dd84700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fd638112770 con 0x7fd6381084d0 2024-05-03T18:46:10.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:09.668+0000 7fd6377fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd6381026c0 0x7fd638114a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:10.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:09.668+0000 7fd6377fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd6381026c0 0x7fd638114a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55332/0 (socket says 172.21.15.32:55332) 2024-05-03T18:46:10.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:09.668+0000 7fd6377fe700 1 -- 172.21.15.32:0/1616850784 learned_addr learned my addr 172.21.15.32:0/1616850784 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:46:10.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:09.668+0000 7fd637fff700 1 --2- 172.21.15.32:0/1616850784 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd6381084d0 0x7fd638113520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:10.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:09.668+0000 7fd636ffd700 1 --2- 172.21.15.32:0/1616850784 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd638103060 0x7fd638112db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:10.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:09.668+0000 7fd6377fe700 1 -- 172.21.15.32:0/1616850784 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd6381084d0 msgr2=0x7fd638113520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:10.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:09.668+0000 7fd6377fe700 1 --2- 172.21.15.32:0/1616850784 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd6381084d0 0x7fd638113520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:10.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:09.668+0000 7fd6377fe700 1 -- 172.21.15.32:0/1616850784 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd638103060 msgr2=0x7fd638112db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:10.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:09.668+0000 7fd6377fe700 1 --2- 172.21.15.32:0/1616850784 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd638103060 0x7fd638112db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:10.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:09.668+0000 7fd6377fe700 1 -- 172.21.15.32:0/1616850784 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6381a8150 con 0x7fd6381026c0 2024-05-03T18:46:10.705 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:09.668+0000 7fd637fff700 1 --2- 172.21.15.32:0/1616850784 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd6381084d0 0x7fd638113520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T18:46:10.706 INFO:teuthology.orchestra.run.smithi032.stdout:274877906980 2024-05-03T18:46:10.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:09.668+0000 7fd6377fe700 1 --2- 172.21.15.32:0/1616850784 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd6381026c0 0x7fd638114a10 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fd62800b530 tx=0x7fd62800b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:46:10.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:09.669+0000 7fd634ff9700 1 -- 172.21.15.32:0/1616850784 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd628015020 con 0x7fd6381026c0 2024-05-03T18:46:10.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:09.669+0000 7fd63dd84700 1 -- 172.21.15.32:0/1616850784 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6381a83e0 con 0x7fd6381026c0 2024-05-03T18:46:10.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:09.669+0000 7fd634ff9700 1 -- 172.21.15.32:0/1616850784 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd628011ce0 con 0x7fd6381026c0 2024-05-03T18:46:10.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:09.669+0000 7fd634ff9700 1 -- 172.21.15.32:0/1616850784 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd628010d20 con 0x7fd6381026c0 2024-05-03T18:46:10.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:09.669+0000 7fd63dd84700 1 -- 172.21.15.32:0/1616850784 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6381a8940 con 0x7fd6381026c0 2024-05-03T18:46:10.706 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:09.670+0000 7fd63dd84700 1 -- 172.21.15.32:0/1616850784 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd63806a190 con 0x7fd6381026c0 2024-05-03T18:46:10.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:09.671+0000 7fd634ff9700 1 -- 172.21.15.32:0/1616850784 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fd628018020 con 0x7fd6381026c0 2024-05-03T18:46:10.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:09.671+0000 7fd634ff9700 1 --2- 172.21.15.32:0/1616850784 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fd6240b2910 0x7fd6240b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:10.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:09.672+0000 7fd636ffd700 1 --2- 172.21.15.32:0/1616850784 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fd6240b2910 0x7fd6240b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:10.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:09.672+0000 7fd634ff9700 1 -- 172.21.15.32:0/1616850784 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8269+0+0 (secure 0 0 0) 0x7fd6280e20c0 con 0x7fd6381026c0 2024-05-03T18:46:10.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:09.672+0000 7fd636ffd700 1 --2- 172.21.15.32:0/1616850784 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fd6240b2910 0x7fd6240b4dd0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fd620005950 tx=0x7fd6200058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:46:10.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:09.675+0000 7fd634ff9700 1 -- 172.21.15.32:0/1616850784 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fd62801f020 con 0x7fd6381026c0 2024-05-03T18:46:10.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:09.902+0000 7fd63dd84700 1 -- 172.21.15.32:0/1616850784 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7fd6381a8c00 con 0x7fd6381026c0 2024-05-03T18:46:10.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:09.902+0000 7fd634ff9700 1 -- 172.21.15.32:0/1616850784 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fd6280a9d70 con 0x7fd6381026c0 2024-05-03T18:46:10.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:09.905+0000 7fd63dd84700 1 -- 172.21.15.32:0/1616850784 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fd6240b2910 msgr2=0x7fd6240b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:10.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:09.905+0000 7fd63dd84700 1 --2- 172.21.15.32:0/1616850784 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fd6240b2910 0x7fd6240b4dd0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fd620005950 tx=0x7fd6200058e0 comp rx=0 tx=0).stop 2024-05-03T18:46:10.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:09.905+0000 7fd63dd84700 1 -- 172.21.15.32:0/1616850784 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd6381026c0 msgr2=0x7fd638114a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:10.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:09.905+0000 7fd63dd84700 1 --2- 172.21.15.32:0/1616850784 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd6381026c0 0x7fd638114a10 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fd62800b530 tx=0x7fd62800b8f0 comp rx=0 tx=0).stop 2024-05-03T18:46:10.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:09.905+0000 7fd63dd84700 1 -- 172.21.15.32:0/1616850784 shutdown_connections 2024-05-03T18:46:10.707 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:09.905+0000 7fd63dd84700 1 --2- 172.21.15.32:0/1616850784 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fd6240b2910 0x7fd6240b4dd0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:10.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:09.905+0000 7fd63dd84700 1 --2- 172.21.15.32:0/1616850784 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd6381084d0 0x7fd638113520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:10.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:09.905+0000 7fd63dd84700 1 --2- 172.21.15.32:0/1616850784 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd6381026c0 0x7fd638114a10 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:10.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:09.905+0000 7fd63dd84700 1 --2- 172.21.15.32:0/1616850784 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd638103060 0x7fd638112db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:10.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:09.905+0000 7fd63dd84700 1 -- 172.21.15.32:0/1616850784 >> 172.21.15.32:0/1616850784 conn(0x7fd638079170 msgr2=0x7fd638100710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:46:10.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:09.905+0000 7fd63dd84700 1 -- 172.21.15.32:0/1616850784 shutdown_connections 2024-05-03T18:46:10.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:09.905+0000 7fd63dd84700 1 -- 172.21.15.32:0/1616850784 wait complete. 2024-05-03T18:46:11.183 INFO:tasks.cephadm.ceph_manager.ceph:need seq 274877906976 got 274877906980 for osd.10 2024-05-03T18:46:11.183 DEBUG:teuthology.parallel:result is None 2024-05-03T18:46:11.308 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:46:11 smithi032 ceph-mon[29763]: pgmap v322: 97 pgs: 97 active+clean; 631 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-05-03T18:46:11.497 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:46:11 smithi045 ceph-mon[33299]: pgmap v322: 97 pgs: 97 active+clean; 631 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-05-03T18:46:11.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:46:11 smithi113 ceph-mon[33540]: pgmap v322: 97 pgs: 97 active+clean; 631 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-05-03T18:46:11.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:11.916+0000 7fe7679d2700 1 -- 172.21.15.32:0/82173306 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe760108410 msgr2=0x7fe7601087f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:11.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:11.916+0000 7fe7679d2700 1 --2- 172.21.15.32:0/82173306 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe760108410 0x7fe7601087f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fe75c009990 tx=0x7fe75c009ca0 comp rx=0 tx=0).stop 2024-05-03T18:46:11.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:11.916+0000 7fe7679d2700 1 -- 172.21.15.32:0/82173306 shutdown_connections 2024-05-03T18:46:11.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:11.916+0000 7fe7679d2700 1 --2- 172.21.15.32:0/82173306 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe760102d60 0x7fe76010f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:11.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:11.916+0000 7fe7679d2700 1 --2- 172.21.15.32:0/82173306 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe760108410 0x7fe7601087f0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:11.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:11.916+0000 7fe7679d2700 1 --2- 172.21.15.32:0/82173306 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe7601023a0 0x7fe760102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:11.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:11.916+0000 7fe7679d2700 1 -- 172.21.15.32:0/82173306 >> 172.21.15.32:0/82173306 conn(0x7fe760079350 msgr2=0x7fe760079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:46:11.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:11.916+0000 7fe7679d2700 1 -- 172.21.15.32:0/82173306 shutdown_connections 2024-05-03T18:46:11.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:11.916+0000 7fe7679d2700 1 -- 172.21.15.32:0/82173306 wait complete. 2024-05-03T18:46:11.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:11.917+0000 7fe7679d2700 1 Processor -- start 2024-05-03T18:46:11.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:11.917+0000 7fe7679d2700 1 -- start start 2024-05-03T18:46:11.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:11.918+0000 7fe7679d2700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe7601023a0 0x7fe76019f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:11.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:11.918+0000 7fe7679d2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe760102d60 0x7fe76019fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:11.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:11.918+0000 7fe7679d2700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe760108410 0x7fe760199a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:11.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:11.918+0000 7fe7679d2700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe760112600 con 0x7fe760102d60 2024-05-03T18:46:11.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:11.918+0000 7fe7679d2700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe760112480 con 0x7fe760108410 2024-05-03T18:46:11.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:11.918+0000 7fe7679d2700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fe760112780 con 0x7fe7601023a0 2024-05-03T18:46:11.920 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:11.918+0000 7fe764f6d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe760102d60 0x7fe76019fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:11.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:11.918+0000 7fe764f6d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe760102d60 0x7fe76019fdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55342/0 (socket says 172.21.15.32:55342) 2024-05-03T18:46:11.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:11.918+0000 7fe764f6d700 1 -- 172.21.15.32:0/1717788998 learned_addr learned my addr 172.21.15.32:0/1717788998 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:46:11.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:11.918+0000 7fe765f6f700 1 --2- 172.21.15.32:0/1717788998 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe760108410 0x7fe760199a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:11.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:11.918+0000 7fe76576e700 1 --2- 172.21.15.32:0/1717788998 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe7601023a0 0x7fe76019f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:11.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:11.918+0000 7fe764f6d700 1 -- 172.21.15.32:0/1717788998 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe7601023a0 msgr2=0x7fe76019f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:11.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:11.918+0000 7fe764f6d700 1 --2- 172.21.15.32:0/1717788998 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe7601023a0 0x7fe76019f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:11.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:11.918+0000 7fe764f6d700 1 -- 172.21.15.32:0/1717788998 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe760108410 msgr2=0x7fe760199a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:11.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:11.918+0000 7fe764f6d700 1 --2- 172.21.15.32:0/1717788998 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe760108410 0x7fe760199a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:11.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:11.918+0000 7fe764f6d700 1 -- 172.21.15.32:0/1717788998 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe76019a1b0 con 0x7fe760102d60 2024-05-03T18:46:11.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:11.919+0000 7fe765f6f700 1 --2- 172.21.15.32:0/1717788998 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe760108410 0x7fe760199a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T18:46:11.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:11.919+0000 7fe764f6d700 1 --2- 172.21.15.32:0/1717788998 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe760102d60 0x7fe76019fdd0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fe75000b530 tx=0x7fe75000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:46:11.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:11.919+0000 7fe7567fc700 1 -- 172.21.15.32:0/1717788998 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe750015020 con 0x7fe760102d60 2024-05-03T18:46:11.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:11.919+0000 7fe7567fc700 1 -- 172.21.15.32:0/1717788998 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe750011ce0 con 0x7fe760102d60 2024-05-03T18:46:11.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:11.919+0000 7fe7679d2700 1 -- 172.21.15.32:0/1717788998 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe76019a4a0 con 0x7fe760102d60 2024-05-03T18:46:11.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:11.919+0000 7fe7567fc700 1 -- 172.21.15.32:0/1717788998 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe750010e40 con 0x7fe760102d60 2024-05-03T18:46:11.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:11.920+0000 7fe7679d2700 1 -- 172.21.15.32:0/1717788998 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe760103fb0 con 0x7fe760102d60 2024-05-03T18:46:11.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:11.921+0000 7fe7679d2700 1 -- 172.21.15.32:0/1717788998 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe760107520 con 0x7fe760102d60 2024-05-03T18:46:11.923 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:11.922+0000 7fe7567fc700 1 -- 172.21.15.32:0/1717788998 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fe750018020 con 0x7fe760102d60 2024-05-03T18:46:11.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:11.923+0000 7fe7567fc700 1 --2- 172.21.15.32:0/1717788998 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fe74c0b2990 0x7fe74c0b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:11.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:11.923+0000 7fe76576e700 1 --2- 172.21.15.32:0/1717788998 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fe74c0b2990 0x7fe74c0b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:11.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:11.923+0000 7fe7567fc700 1 -- 172.21.15.32:0/1717788998 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8269+0+0 (secure 0 0 0) 0x7fe7500e1a50 con 0x7fe760102d60 2024-05-03T18:46:11.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:11.923+0000 7fe7567fc700 1 --2- 172.21.15.32:0/1717788998 >> [v2:172.21.15.32:6826/4290196345,v1:172.21.15.32:6827/4290196345] conn(0x7fe74c0bc5d0 0x7fe74c0bea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:11.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:11.923+0000 7fe7567fc700 1 -- 172.21.15.32:0/1717788998 --> [v2:172.21.15.32:6826/4290196345,v1:172.21.15.32:6827/4290196345] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe74c0bf140 con 0x7fe74c0bc5d0 2024-05-03T18:46:11.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:11.923+0000 7fe7567fc700 1 -- 172.21.15.32:0/1717788998 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7fe7500e62c0 con 0x7fe760102d60 2024-05-03T18:46:11.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:11.923+0000 7fe765f6f700 1 --2- 172.21.15.32:0/1717788998 >> [v2:172.21.15.32:6826/4290196345,v1:172.21.15.32:6827/4290196345] conn(0x7fe74c0bc5d0 0x7fe74c0bea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:11.924 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:11.923+0000 7fe76576e700 1 --2- 172.21.15.32:0/1717788998 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fe74c0b2990 0x7fe74c0b4e50 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fe75c00b5c0 tx=0x7fe75c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:46:11.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:11.923+0000 7fe765f6f700 1 --2- 172.21.15.32:0/1717788998 >> [v2:172.21.15.32:6826/4290196345,v1:172.21.15.32:6827/4290196345] conn(0x7fe74c0bc5d0 0x7fe74c0bea70 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:46:11.925 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:11.924+0000 7fe7567fc700 1 -- 172.21.15.32:0/1717788998 <== osd.9 v2:172.21.15.32:6826/4290196345 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe74c0bf140 con 0x7fe74c0bc5d0 2024-05-03T18:46:11.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:11.941+0000 7fe7679d2700 1 -- 172.21.15.32:0/1717788998 --> [v2:172.21.15.32:6826/4290196345,v1:172.21.15.32:6827/4290196345] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe760103ac0 con 0x7fe74c0bc5d0 2024-05-03T18:46:11.942 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:11.941+0000 7fe7567fc700 1 -- 172.21.15.32:0/1717788998 <== osd.9 v2:172.21.15.32:6826/4290196345 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fe760103ac0 con 0x7fe74c0bc5d0 2024-05-03T18:46:11.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:11.941+0000 7fe7679d2700 1 -- 172.21.15.32:0/1717788998 >> [v2:172.21.15.32:6826/4290196345,v1:172.21.15.32:6827/4290196345] conn(0x7fe74c0bc5d0 msgr2=0x7fe74c0bea70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:11.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:11.941+0000 7fe7679d2700 1 --2- 172.21.15.32:0/1717788998 >> [v2:172.21.15.32:6826/4290196345,v1:172.21.15.32:6827/4290196345] conn(0x7fe74c0bc5d0 0x7fe74c0bea70 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:11.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:11.942+0000 7fe7679d2700 1 -- 172.21.15.32:0/1717788998 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fe74c0b2990 msgr2=0x7fe74c0b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:11.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:11.942+0000 7fe7679d2700 1 --2- 172.21.15.32:0/1717788998 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fe74c0b2990 0x7fe74c0b4e50 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fe75c00b5c0 tx=0x7fe75c005fb0 comp rx=0 tx=0).stop 2024-05-03T18:46:11.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:11.942+0000 7fe7679d2700 1 -- 172.21.15.32:0/1717788998 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe760102d60 msgr2=0x7fe76019fdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:11.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:11.942+0000 7fe7679d2700 1 --2- 172.21.15.32:0/1717788998 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe760102d60 0x7fe76019fdd0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fe75000b530 tx=0x7fe75000b840 comp rx=0 tx=0).stop 2024-05-03T18:46:11.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:11.942+0000 7fe765f6f700 1 -- 172.21.15.32:0/1717788998 reap_dead start 2024-05-03T18:46:11.943 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:11.942+0000 7fe7679d2700 1 -- 172.21.15.32:0/1717788998 shutdown_connections 2024-05-03T18:46:11.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:11.942+0000 7fe7679d2700 1 -- 172.21.15.32:0/1717788998 >> 172.21.15.32:0/1717788998 conn(0x7fe760079350 msgr2=0x7fe76006b320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:46:11.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:11.942+0000 7fe7679d2700 1 -- 172.21.15.32:0/1717788998 shutdown_connections 2024-05-03T18:46:11.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:11.942+0000 7fe7679d2700 1 -- 172.21.15.32:0/1717788998 wait complete. 2024-05-03T18:46:12.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:12.124+0000 7fe4a79f6700 1 -- 172.21.15.32:0/2588378861 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4a006c590 msgr2=0x7fe4a0110330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:12.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:12.124+0000 7fe4a79f6700 1 --2- 172.21.15.32:0/2588378861 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4a006c590 0x7fe4a0110330 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fe49800b0a0 tx=0x7fe49800b3b0 comp rx=0 tx=0).stop 2024-05-03T18:46:12.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:12.125+0000 7fe4a79f6700 1 -- 172.21.15.32:0/2588378861 shutdown_connections 2024-05-03T18:46:12.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:12.125+0000 7fe4a79f6700 1 --2- 172.21.15.32:0/2588378861 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4a006c590 0x7fe4a0110330 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:12.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:12.125+0000 7fe4a79f6700 1 --2- 172.21.15.32:0/2588378861 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe4a0109510 0x7fe4a01098f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:12.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:12.125+0000 7fe4a79f6700 1 --2- 172.21.15.32:0/2588378861 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe4a006bbd0 0x7fe4a006c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:12.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:12.125+0000 7fe4a79f6700 1 -- 172.21.15.32:0/2588378861 >> 172.21.15.32:0/2588378861 conn(0x7fe4a0079350 msgr2=0x7fe4a0079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:46:12.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:12.125+0000 7fe4a79f6700 1 -- 172.21.15.32:0/2588378861 shutdown_connections 2024-05-03T18:46:12.126 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:12.125+0000 7fe4a79f6700 1 -- 172.21.15.32:0/2588378861 wait complete. 2024-05-03T18:46:12.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:12.125+0000 7fe4a79f6700 1 Processor -- start 2024-05-03T18:46:12.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:12.125+0000 7fe4a79f6700 1 -- start start 2024-05-03T18:46:12.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:12.125+0000 7fe4a79f6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4a006bbd0 0x7fe4a019f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:12.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:12.126+0000 7fe4a79f6700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe4a006c590 0x7fe4a019fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:12.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:12.126+0000 7fe4a79f6700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe4a0109510 0x7fe4a0199880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:12.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:12.126+0000 7fe4a79f6700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fe4a0113620 con 0x7fe4a006bbd0 2024-05-03T18:46:12.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:12.126+0000 7fe4a79f6700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe4a01134a0 con 0x7fe4a0109510 2024-05-03T18:46:12.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:12.126+0000 7fe4a79f6700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fe4a01137a0 con 0x7fe4a006c590 2024-05-03T18:46:12.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:12.126+0000 7fe4a5792700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4a006bbd0 0x7fe4a019f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:12.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:12.126+0000 7fe4a5792700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4a006bbd0 0x7fe4a019f6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55374/0 (socket says 172.21.15.32:55374) 2024-05-03T18:46:12.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:12.126+0000 7fe4a5792700 1 -- 172.21.15.32:0/4068704759 learned_addr learned my addr 172.21.15.32:0/4068704759 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:46:12.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:12.126+0000 7fe4a4f91700 1 --2- 172.21.15.32:0/4068704759 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe4a006c590 0x7fe4a019fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:12.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:12.126+0000 7fe4a5f93700 1 --2- 172.21.15.32:0/4068704759 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe4a0109510 0x7fe4a0199880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:12.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:12.126+0000 7fe4a5792700 1 -- 172.21.15.32:0/4068704759 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe4a006c590 msgr2=0x7fe4a019fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:12.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:12.126+0000 7fe4a5792700 1 --2- 172.21.15.32:0/4068704759 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe4a006c590 0x7fe4a019fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:12.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:12.126+0000 7fe4a5792700 1 -- 172.21.15.32:0/4068704759 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe4a0109510 msgr2=0x7fe4a0199880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:12.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:12.126+0000 7fe4a5792700 1 --2- 172.21.15.32:0/4068704759 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe4a0109510 0x7fe4a0199880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:12.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:12.126+0000 7fe4a5792700 1 -- 172.21.15.32:0/4068704759 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4a0199fe0 con 0x7fe4a006bbd0 2024-05-03T18:46:12.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:12.126+0000 7fe4a4f91700 1 --2- 172.21.15.32:0/4068704759 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe4a006c590 0x7fe4a019fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T18:46:12.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:12.126+0000 7fe4a5792700 1 --2- 172.21.15.32:0/4068704759 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4a006bbd0 0x7fe4a019f6c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fe49c00c9b0 tx=0x7fe49c00cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:46:12.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:12.126+0000 7fe4967fc700 1 -- 172.21.15.32:0/4068704759 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe49c007b00 con 0x7fe4a006bbd0 2024-05-03T18:46:12.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:12.126+0000 7fe4967fc700 1 -- 172.21.15.32:0/4068704759 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe49c007ca0 con 0x7fe4a006bbd0 2024-05-03T18:46:12.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:12.126+0000 7fe4a79f6700 1 -- 172.21.15.32:0/4068704759 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4a019a2d0 con 0x7fe4a006bbd0 2024-05-03T18:46:12.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:12.126+0000 7fe4967fc700 1 -- 172.21.15.32:0/4068704759 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe49c018730 con 0x7fe4a006bbd0 2024-05-03T18:46:12.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:12.126+0000 7fe4a79f6700 1 -- 172.21.15.32:0/4068704759 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4a01a6590 con 0x7fe4a006bbd0 2024-05-03T18:46:12.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:12.127+0000 7fe4a79f6700 1 -- 172.21.15.32:0/4068704759 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe4a0063c30 con 0x7fe4a006bbd0 2024-05-03T18:46:12.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:12.129+0000 7fe4967fc700 1 -- 172.21.15.32:0/4068704759 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fe49c00f450 con 0x7fe4a006bbd0 2024-05-03T18:46:12.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:12.130+0000 7fe4967fc700 1 --2- 172.21.15.32:0/4068704759 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fe48c0b2910 0x7fe48c0b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:12.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:12.130+0000 7fe4a4f91700 1 --2- 172.21.15.32:0/4068704759 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fe48c0b2910 0x7fe48c0b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:12.131 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:12.130+0000 7fe4967fc700 1 -- 172.21.15.32:0/4068704759 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8269+0+0 (secure 0 0 0) 0x7fe49c0e2820 con 0x7fe4a006bbd0 2024-05-03T18:46:12.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:12.130+0000 7fe4967fc700 1 --2- 172.21.15.32:0/4068704759 >> [v2:172.21.15.113:6808/4085018597,v1:172.21.15.113:6809/4085018597] conn(0x7fe48c0bc550 0x7fe48c0be9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:12.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:12.130+0000 7fe4967fc700 1 -- 172.21.15.32:0/4068704759 --> [v2:172.21.15.113:6808/4085018597,v1:172.21.15.113:6809/4085018597] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe49c007e50 con 0x7fe48c0bc550 2024-05-03T18:46:12.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:12.130+0000 7fe4967fc700 1 -- 172.21.15.32:0/4068704759 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7fe49c0e2c70 con 0x7fe4a006bbd0 2024-05-03T18:46:12.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:12.130+0000 7fe4a4f91700 1 --2- 172.21.15.32:0/4068704759 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fe48c0b2910 0x7fe48c0b4dd0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fe490005950 tx=0x7fe4900058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:46:12.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:12.130+0000 7fe4a5f93700 1 --2- 172.21.15.32:0/4068704759 >> [v2:172.21.15.113:6808/4085018597,v1:172.21.15.113:6809/4085018597] conn(0x7fe48c0bc550 0x7fe48c0be9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:12.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:12.131+0000 7fe4a5f93700 1 --2- 172.21.15.32:0/4068704759 >> [v2:172.21.15.113:6808/4085018597,v1:172.21.15.113:6809/4085018597] conn(0x7fe48c0bc550 0x7fe48c0be9f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:46:12.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:12.132+0000 7fe4967fc700 1 -- 172.21.15.32:0/4068704759 <== osd.5 v2:172.21.15.113:6808/4085018597 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe49c007e50 con 0x7fe48c0bc550 2024-05-03T18:46:12.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:12.148+0000 7fe4a79f6700 1 -- 172.21.15.32:0/4068704759 --> [v2:172.21.15.113:6808/4085018597,v1:172.21.15.113:6809/4085018597] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe4a0064030 con 0x7fe48c0bc550 2024-05-03T18:46:12.149 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:12.149+0000 7fe4967fc700 1 -- 172.21.15.32:0/4068704759 <== osd.5 v2:172.21.15.113:6808/4085018597 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fe4a0064030 con 0x7fe48c0bc550 2024-05-03T18:46:12.150 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:12.149+0000 7fe4a79f6700 1 -- 172.21.15.32:0/4068704759 >> [v2:172.21.15.113:6808/4085018597,v1:172.21.15.113:6809/4085018597] conn(0x7fe48c0bc550 msgr2=0x7fe48c0be9f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:12.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:12.149+0000 7fe4a79f6700 1 --2- 172.21.15.32:0/4068704759 >> [v2:172.21.15.113:6808/4085018597,v1:172.21.15.113:6809/4085018597] conn(0x7fe48c0bc550 0x7fe48c0be9f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:12.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:12.149+0000 7fe4a79f6700 1 -- 172.21.15.32:0/4068704759 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fe48c0b2910 msgr2=0x7fe48c0b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:12.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:12.149+0000 7fe4a79f6700 1 --2- 172.21.15.32:0/4068704759 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fe48c0b2910 0x7fe48c0b4dd0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fe490005950 tx=0x7fe4900058e0 comp rx=0 tx=0).stop 2024-05-03T18:46:12.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:12.149+0000 7fe4a79f6700 1 -- 172.21.15.32:0/4068704759 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4a006bbd0 msgr2=0x7fe4a019f6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:12.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:12.149+0000 7fe4a79f6700 1 --2- 172.21.15.32:0/4068704759 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4a006bbd0 0x7fe4a019f6c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fe49c00c9b0 tx=0x7fe49c00cd70 comp rx=0 tx=0).stop 2024-05-03T18:46:12.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:12.149+0000 7fe4a5f93700 1 -- 172.21.15.32:0/4068704759 reap_dead start 2024-05-03T18:46:12.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:12.149+0000 7fe4a79f6700 1 -- 172.21.15.32:0/4068704759 shutdown_connections 2024-05-03T18:46:12.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:12.149+0000 7fe4a79f6700 1 -- 172.21.15.32:0/4068704759 >> 172.21.15.32:0/4068704759 conn(0x7fe4a0079350 msgr2=0x7fe4a0101070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:46:12.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:12.149+0000 7fe4a79f6700 1 -- 172.21.15.32:0/4068704759 shutdown_connections 2024-05-03T18:46:12.151 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:12.149+0000 7fe4a79f6700 1 -- 172.21.15.32:0/4068704759 wait complete. 2024-05-03T18:46:12.457 INFO:teuthology.orchestra.run.smithi032.stdout:77309411403 2024-05-03T18:46:12.458 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid f500455e-097b-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-05-03T18:46:12.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:12.630+0000 7f76b01dd700 1 -- 172.21.15.32:0/3834030108 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76a8103540 msgr2=0x7f76a81039c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:12.631 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:12.630+0000 7f76b01dd700 1 --2- 172.21.15.32:0/3834030108 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76a8103540 0x7f76a81039c0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f76a40099c0 tx=0x7f76a4009cd0 comp rx=0 tx=0).stop 2024-05-03T18:46:12.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:12.630+0000 7f76b01dd700 1 -- 172.21.15.32:0/3834030108 shutdown_connections 2024-05-03T18:46:12.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:12.630+0000 7f76b01dd700 1 --2- 172.21.15.32:0/3834030108 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f76a8103f00 0x7f76a81103b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:12.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:12.630+0000 7f76b01dd700 1 --2- 172.21.15.32:0/3834030108 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76a81095e0 0x7f76a81099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:12.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:12.630+0000 7f76b01dd700 1 --2- 172.21.15.32:0/3834030108 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76a8103540 0x7f76a81039c0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:12.632 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:12.630+0000 7f76b01dd700 1 -- 172.21.15.32:0/3834030108 >> 172.21.15.32:0/3834030108 conn(0x7f76a807a650 msgr2=0x7f76a807aa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:46:12.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:12.631+0000 7f76b01dd700 1 -- 172.21.15.32:0/3834030108 shutdown_connections 2024-05-03T18:46:12.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:12.631+0000 7f76b01dd700 1 -- 172.21.15.32:0/3834030108 wait complete. 2024-05-03T18:46:12.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:12.632+0000 7f76b01dd700 1 Processor -- start 2024-05-03T18:46:12.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:12.632+0000 7f76b01dd700 1 -- start start 2024-05-03T18:46:12.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:12.632+0000 7f76b01dd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76a8103540 0x7f76a819f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:12.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:12.632+0000 7f76b01dd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76a8103f00 0x7f76a819fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:12.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:12.632+0000 7f76b01dd700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f76a81095e0 0x7f76a81999a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:12.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:12.632+0000 7f76b01dd700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f76a8113740 con 0x7f76a8103540 2024-05-03T18:46:12.634 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:12.632+0000 7f76b01dd700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f76a81135c0 con 0x7f76a8103f00 2024-05-03T18:46:12.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:12.632+0000 7f76b01dd700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f76a81138c0 con 0x7f76a81095e0 2024-05-03T18:46:12.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:12.632+0000 7f76adf79700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76a8103540 0x7f76a819f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:12.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:12.632+0000 7f76adf79700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76a8103540 0x7f76a819f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55396/0 (socket says 172.21.15.32:55396) 2024-05-03T18:46:12.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:12.632+0000 7f76adf79700 1 -- 172.21.15.32:0/1424150712 learned_addr learned my addr 172.21.15.32:0/1424150712 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:46:12.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:12.632+0000 7f76ad778700 1 --2- 172.21.15.32:0/1424150712 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76a8103f00 0x7f76a819fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:12.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:12.632+0000 7f76ae77a700 1 --2- 172.21.15.32:0/1424150712 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f76a81095e0 0x7f76a81999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:12.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:12.632+0000 7f76adf79700 1 -- 172.21.15.32:0/1424150712 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f76a81095e0 msgr2=0x7f76a81999a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:12.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:12.632+0000 7f76adf79700 1 --2- 172.21.15.32:0/1424150712 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f76a81095e0 0x7f76a81999a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:12.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:12.632+0000 7f76adf79700 1 -- 172.21.15.32:0/1424150712 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76a8103f00 msgr2=0x7f76a819fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:12.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:12.632+0000 7f76adf79700 1 --2- 172.21.15.32:0/1424150712 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76a8103f00 0x7f76a819fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:12.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:12.632+0000 7f76adf79700 1 -- 172.21.15.32:0/1424150712 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76a819a100 con 0x7f76a8103540 2024-05-03T18:46:12.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:12.632+0000 7f76ae77a700 1 --2- 172.21.15.32:0/1424150712 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f76a81095e0 0x7f76a81999a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:46:12.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:12.633+0000 7f76adf79700 1 --2- 172.21.15.32:0/1424150712 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76a8103540 0x7f76a819f790 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f769c00e9a0 tx=0x7f769c00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:46:12.635 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:12.633+0000 7f769affd700 1 -- 172.21.15.32:0/1424150712 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f769c00cbd0 con 0x7f76a8103540 2024-05-03T18:46:12.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:12.633+0000 7f76b01dd700 1 -- 172.21.15.32:0/1424150712 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76a819a3f0 con 0x7f76a8103540 2024-05-03T18:46:12.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:12.633+0000 7f76b01dd700 1 -- 172.21.15.32:0/1424150712 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76a81a6660 con 0x7f76a8103540 2024-05-03T18:46:12.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:12.633+0000 7f769affd700 1 -- 172.21.15.32:0/1424150712 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f769c00cd70 con 0x7f76a8103540 2024-05-03T18:46:12.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:12.633+0000 7f769affd700 1 -- 172.21.15.32:0/1424150712 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f769c018680 con 0x7f76a8103540 2024-05-03T18:46:12.636 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:12.634+0000 7f76b01dd700 1 -- 172.21.15.32:0/1424150712 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76a8104c50 con 0x7f76a8103540 2024-05-03T18:46:12.637 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:12.636+0000 7f769affd700 1 -- 172.21.15.32:0/1424150712 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f769c010970 con 0x7f76a8103540 2024-05-03T18:46:12.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:12.637+0000 7f769affd700 1 --2- 172.21.15.32:0/1424150712 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f76940b2990 0x7f76940b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:12.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:12.637+0000 7f76ad778700 1 --2- 172.21.15.32:0/1424150712 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f76940b2990 0x7f76940b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:12.638 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:12.637+0000 7f769affd700 1 -- 172.21.15.32:0/1424150712 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8269+0+0 (secure 0 0 0) 0x7f769c0e2bc0 con 0x7f76a8103540 2024-05-03T18:46:12.639 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:12.638+0000 7f76ad778700 1 --2- 172.21.15.32:0/1424150712 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f76940b2990 0x7f76940b4e50 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f76a807ac50 tx=0x7f76a4005940 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:46:12.642 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:12.642+0000 7f769affd700 1 -- 172.21.15.32:0/1424150712 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f769c0aa700 con 0x7f76a8103540 2024-05-03T18:46:12.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:12.865+0000 7f76b01dd700 1 -- 172.21.15.32:0/1424150712 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f76a819ae80 con 0x7f76a8103540 2024-05-03T18:46:12.866 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:12.865+0000 7f769affd700 1 -- 172.21.15.32:0/1424150712 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f769c0aa000 con 0x7f76a8103540 2024-05-03T18:46:12.866 INFO:teuthology.orchestra.run.smithi032.stdout:201863462964 2024-05-03T18:46:12.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:12.867+0000 7f76b01dd700 1 -- 172.21.15.32:0/1424150712 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f76940b2990 msgr2=0x7f76940b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:12.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:12.867+0000 7f76b01dd700 1 --2- 172.21.15.32:0/1424150712 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f76940b2990 0x7f76940b4e50 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f76a807ac50 tx=0x7f76a4005940 comp rx=0 tx=0).stop 2024-05-03T18:46:12.868 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:12.867+0000 7f76b01dd700 1 -- 172.21.15.32:0/1424150712 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76a8103540 msgr2=0x7f76a819f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:12.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:12.867+0000 7f76b01dd700 1 --2- 172.21.15.32:0/1424150712 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76a8103540 0x7f76a819f790 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f769c00e9a0 tx=0x7f769c00ed60 comp rx=0 tx=0).stop 2024-05-03T18:46:12.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:12.867+0000 7f76b01dd700 1 -- 172.21.15.32:0/1424150712 shutdown_connections 2024-05-03T18:46:12.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:12.867+0000 7f76b01dd700 1 --2- 172.21.15.32:0/1424150712 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f76940b2990 0x7f76940b4e50 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:12.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:12.867+0000 7f76b01dd700 1 --2- 172.21.15.32:0/1424150712 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f76a81095e0 0x7f76a81999a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:12.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:12.867+0000 7f76b01dd700 1 --2- 172.21.15.32:0/1424150712 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76a8103540 0x7f76a819f790 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:12.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:12.867+0000 7f76b01dd700 1 --2- 172.21.15.32:0/1424150712 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f76a8103f00 0x7f76a819fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:12.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:12.867+0000 7f76b01dd700 1 -- 172.21.15.32:0/1424150712 >> 172.21.15.32:0/1424150712 conn(0x7f76a807a650 msgr2=0x7f76a8102240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:46:12.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:12.867+0000 7f76b01dd700 1 -- 172.21.15.32:0/1424150712 shutdown_connections 2024-05-03T18:46:12.869 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:12.867+0000 7f76b01dd700 1 -- 172.21.15.32:0/1424150712 wait complete. 2024-05-03T18:46:13.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:46:13 smithi032 ceph-mon[29763]: pgmap v323: 97 pgs: 97 active+clean; 631 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-05-03T18:46:13.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:46:13 smithi032 ceph-mon[29763]: from='client.? 172.21.15.32:0/1424150712' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-03T18:46:13.497 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:46:13 smithi045 ceph-mon[33299]: pgmap v323: 97 pgs: 97 active+clean; 631 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-05-03T18:46:13.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:46:13 smithi045 ceph-mon[33299]: from='client.? 172.21.15.32:0/1424150712' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-03T18:46:13.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:46:13 smithi113 ceph-mon[33540]: pgmap v323: 97 pgs: 97 active+clean; 631 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-05-03T18:46:13.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:46:13 smithi113 ceph-mon[33540]: from='client.? 172.21.15.32:0/1424150712' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-03T18:46:14.462 INFO:teuthology.orchestra.run.smithi032.stdout:154618822717 2024-05-03T18:46:14.462 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid f500455e-097b-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-05-03T18:46:14.899 INFO:teuthology.orchestra.run.smithi032.stdout:253403070505 2024-05-03T18:46:14.899 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid f500455e-097b-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-05-03T18:46:15.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:15.103+0000 7f78af322700 1 -- 172.21.15.32:0/3532393972 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f78a806bc70 msgr2=0x7f78a806c0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:15.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:15.103+0000 7f78af322700 1 --2- 172.21.15.32:0/3532393972 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f78a806bc70 0x7f78a806c0f0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f78980099c0 tx=0x7f7898009cd0 comp rx=0 tx=0).stop 2024-05-03T18:46:15.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:15.103+0000 7f78af322700 1 -- 172.21.15.32:0/3532393972 shutdown_connections 2024-05-03T18:46:15.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:15.103+0000 7f78af322700 1 --2- 172.21.15.32:0/3532393972 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f78a806c630 0x7f78a8110360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:15.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:15.103+0000 7f78af322700 1 --2- 172.21.15.32:0/3532393972 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f78a8109590 0x7f78a8109970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:15.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:15.103+0000 7f78af322700 1 --2- 172.21.15.32:0/3532393972 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f78a806bc70 0x7f78a806c0f0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:15.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:15.103+0000 7f78af322700 1 -- 172.21.15.32:0/3532393972 >> 172.21.15.32:0/3532393972 conn(0x7f78a8079440 msgr2=0x7f78a8079850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:46:15.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:15.103+0000 7f78af322700 1 -- 172.21.15.32:0/3532393972 shutdown_connections 2024-05-03T18:46:15.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:15.103+0000 7f78af322700 1 -- 172.21.15.32:0/3532393972 wait complete. 2024-05-03T18:46:15.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:15.103+0000 7f78af322700 1 Processor -- start 2024-05-03T18:46:15.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:15.103+0000 7f78af322700 1 -- start start 2024-05-03T18:46:15.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:15.104+0000 7f78af322700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f78a806bc70 0x7f78a8102410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:15.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:15.104+0000 7f78af322700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f78a806c630 0x7f78a8102950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:15.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:15.104+0000 7f78af322700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f78a8109590 0x7f78a8102e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:15.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:15.104+0000 7f78af322700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f78a8113770 con 0x7f78a806bc70 2024-05-03T18:46:15.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:15.104+0000 7f78af322700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f78a81135f0 con 0x7f78a806c630 2024-05-03T18:46:15.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:15.104+0000 7f78af322700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f78a81138f0 con 0x7f78a8109590 2024-05-03T18:46:15.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:15.104+0000 7f78ad0be700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f78a806bc70 0x7f78a8102410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:15.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:15.104+0000 7f78ad0be700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f78a806bc70 0x7f78a8102410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37176/0 (socket says 172.21.15.32:37176) 2024-05-03T18:46:15.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:15.104+0000 7f78ad0be700 1 -- 172.21.15.32:0/3767499511 learned_addr learned my addr 172.21.15.32:0/3767499511 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:46:15.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:15.104+0000 7f78ad8bf700 1 --2- 172.21.15.32:0/3767499511 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f78a8109590 0x7f78a8102e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:15.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:15.104+0000 7f78ac8bd700 1 --2- 172.21.15.32:0/3767499511 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f78a806c630 0x7f78a8102950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:15.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:15.104+0000 7f78ad0be700 1 -- 172.21.15.32:0/3767499511 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f78a8109590 msgr2=0x7f78a8102e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:15.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:15.104+0000 7f78ad0be700 1 --2- 172.21.15.32:0/3767499511 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f78a8109590 0x7f78a8102e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:15.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:15.104+0000 7f78ad0be700 1 -- 172.21.15.32:0/3767499511 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f78a806c630 msgr2=0x7f78a8102950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:15.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:15.104+0000 7f78ad0be700 1 --2- 172.21.15.32:0/3767499511 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f78a806c630 0x7f78a8102950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:15.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:15.104+0000 7f78ad0be700 1 -- 172.21.15.32:0/3767499511 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78a81a8190 con 0x7f78a806bc70 2024-05-03T18:46:15.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:15.104+0000 7f78ad0be700 1 --2- 172.21.15.32:0/3767499511 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f78a806bc70 0x7f78a8102410 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f78a400ba50 tx=0x7f78a400bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:46:15.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:15.104+0000 7f789e7fc700 1 -- 172.21.15.32:0/3767499511 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78a4019070 con 0x7f78a806bc70 2024-05-03T18:46:15.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:15.104+0000 7f789e7fc700 1 -- 172.21.15.32:0/3767499511 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78a4014440 con 0x7f78a806bc70 2024-05-03T18:46:15.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:15.104+0000 7f78af322700 1 -- 172.21.15.32:0/3767499511 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78a81a8480 con 0x7f78a806bc70 2024-05-03T18:46:15.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:15.104+0000 7f789e7fc700 1 -- 172.21.15.32:0/3767499511 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78a400ce60 con 0x7f78a806bc70 2024-05-03T18:46:15.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:15.104+0000 7f78af322700 1 -- 172.21.15.32:0/3767499511 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78a81a89e0 con 0x7f78a806bc70 2024-05-03T18:46:15.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:15.105+0000 7f78af322700 1 -- 172.21.15.32:0/3767499511 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f78a8063c30 con 0x7f78a806bc70 2024-05-03T18:46:15.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:15.107+0000 7f789e7fc700 1 -- 172.21.15.32:0/3767499511 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f78a401c070 con 0x7f78a806bc70 2024-05-03T18:46:15.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:15.108+0000 7f789e7fc700 1 --2- 172.21.15.32:0/3767499511 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f78940b2990 0x7f78940b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:15.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:15.108+0000 7f78ac8bd700 1 --2- 172.21.15.32:0/3767499511 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f78940b2990 0x7f78940b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:15.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:15.108+0000 7f789e7fc700 1 -- 172.21.15.32:0/3767499511 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8269+0+0 (secure 0 0 0) 0x7f78a40e2580 con 0x7f78a806bc70 2024-05-03T18:46:15.115 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:15.108+0000 7f789e7fc700 1 --2- 172.21.15.32:0/3767499511 >> [v2:172.21.15.32:6802/262324956,v1:172.21.15.32:6803/262324956] conn(0x7f78940bc5d0 0x7f78940bea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:15.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:15.108+0000 7f789e7fc700 1 -- 172.21.15.32:0/3767499511 --> [v2:172.21.15.32:6802/262324956,v1:172.21.15.32:6803/262324956] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f78a400aea0 con 0x7f78940bc5d0 2024-05-03T18:46:15.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:15.108+0000 7f789e7fc700 1 -- 172.21.15.32:0/3767499511 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7f78a40e2930 con 0x7f78a806bc70 2024-05-03T18:46:15.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:15.108+0000 7f78ad8bf700 1 --2- 172.21.15.32:0/3767499511 >> [v2:172.21.15.32:6802/262324956,v1:172.21.15.32:6803/262324956] conn(0x7f78940bc5d0 0x7f78940bea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:15.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:15.108+0000 7f78ac8bd700 1 --2- 172.21.15.32:0/3767499511 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f78940b2990 0x7f78940b4e50 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f78a8103ea0 tx=0x7f7898009f10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:46:15.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:15.108+0000 7f78ad8bf700 1 --2- 172.21.15.32:0/3767499511 >> [v2:172.21.15.32:6802/262324956,v1:172.21.15.32:6803/262324956] conn(0x7f78940bc5d0 0x7f78940bea70 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:46:15.116 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:15.109+0000 7f789e7fc700 1 -- 172.21.15.32:0/3767499511 <== osd.0 v2:172.21.15.32:6802/262324956 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f78a400aea0 con 0x7f78940bc5d0 2024-05-03T18:46:15.127 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:15.126+0000 7f78af322700 1 -- 172.21.15.32:0/3767499511 --> [v2:172.21.15.32:6802/262324956,v1:172.21.15.32:6803/262324956] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f78a80640b0 con 0x7f78940bc5d0 2024-05-03T18:46:15.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:15.127+0000 7f789e7fc700 1 -- 172.21.15.32:0/3767499511 <== osd.0 v2:172.21.15.32:6802/262324956 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f78a80640b0 con 0x7f78940bc5d0 2024-05-03T18:46:15.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:15.127+0000 7f78af322700 1 -- 172.21.15.32:0/3767499511 >> [v2:172.21.15.32:6802/262324956,v1:172.21.15.32:6803/262324956] conn(0x7f78940bc5d0 msgr2=0x7f78940bea70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:15.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:15.127+0000 7f78af322700 1 --2- 172.21.15.32:0/3767499511 >> [v2:172.21.15.32:6802/262324956,v1:172.21.15.32:6803/262324956] conn(0x7f78940bc5d0 0x7f78940bea70 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:15.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:15.127+0000 7f78af322700 1 -- 172.21.15.32:0/3767499511 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f78940b2990 msgr2=0x7f78940b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:15.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:15.127+0000 7f78af322700 1 --2- 172.21.15.32:0/3767499511 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f78940b2990 0x7f78940b4e50 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f78a8103ea0 tx=0x7f7898009f10 comp rx=0 tx=0).stop 2024-05-03T18:46:15.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:15.127+0000 7f78af322700 1 -- 172.21.15.32:0/3767499511 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f78a806bc70 msgr2=0x7f78a8102410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:15.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:15.127+0000 7f78af322700 1 --2- 172.21.15.32:0/3767499511 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f78a806bc70 0x7f78a8102410 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f78a400ba50 tx=0x7f78a400bd60 comp rx=0 tx=0).stop 2024-05-03T18:46:15.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:15.127+0000 7f78ad8bf700 1 -- 172.21.15.32:0/3767499511 reap_dead start 2024-05-03T18:46:15.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:15.127+0000 7f78af322700 1 -- 172.21.15.32:0/3767499511 shutdown_connections 2024-05-03T18:46:15.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:15.127+0000 7f78af322700 1 -- 172.21.15.32:0/3767499511 >> 172.21.15.32:0/3767499511 conn(0x7f78a8079440 msgr2=0x7f78a8100fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:46:15.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:15.127+0000 7f78af322700 1 -- 172.21.15.32:0/3767499511 shutdown_connections 2024-05-03T18:46:15.129 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:15.127+0000 7f78af322700 1 -- 172.21.15.32:0/3767499511 wait complete. 2024-05-03T18:46:15.425 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:46:15 smithi032 ceph-mon[29763]: pgmap v324: 97 pgs: 97 active+clean; 631 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:46:15.497 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:46:15 smithi045 ceph-mon[33299]: pgmap v324: 97 pgs: 97 active+clean; 631 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:46:15.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:46:15 smithi113 ceph-mon[33540]: pgmap v324: 97 pgs: 97 active+clean; 631 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:46:15.789 INFO:tasks.cephadm.ceph_manager.ceph:need seq 201863462961 got 201863462964 for osd.7 2024-05-03T18:46:15.790 DEBUG:teuthology.parallel:result is None 2024-05-03T18:46:16.436 INFO:teuthology.orchestra.run.smithi032.stdout:34359738453 2024-05-03T18:46:16.436 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid f500455e-097b-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-05-03T18:46:16.617 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:16.616+0000 7f937a80a700 1 -- 172.21.15.32:0/2915085744 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93741023d0 msgr2=0x7f9374102850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:16.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:16.616+0000 7f937a80a700 1 --2- 172.21.15.32:0/2915085744 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93741023d0 0x7f9374102850 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f935c009990 tx=0x7f935c009ca0 comp rx=0 tx=0).stop 2024-05-03T18:46:16.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:16.616+0000 7f937a80a700 1 -- 172.21.15.32:0/2915085744 shutdown_connections 2024-05-03T18:46:16.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:16.616+0000 7f937a80a700 1 --2- 172.21.15.32:0/2915085744 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9374102d90 0x7f937410f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:16.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:16.616+0000 7f937a80a700 1 --2- 172.21.15.32:0/2915085744 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9374108440 0x7f9374108820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:16.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:16.616+0000 7f937a80a700 1 --2- 172.21.15.32:0/2915085744 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93741023d0 0x7f9374102850 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:16.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:16.616+0000 7f937a80a700 1 -- 172.21.15.32:0/2915085744 >> 172.21.15.32:0/2915085744 conn(0x7f93740793e0 msgr2=0x7f93740797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:46:16.618 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:16.616+0000 7f937a80a700 1 -- 172.21.15.32:0/2915085744 shutdown_connections 2024-05-03T18:46:16.619 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:16.616+0000 7f937a80a700 1 -- 172.21.15.32:0/2915085744 wait complete. 2024-05-03T18:46:16.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:16.617+0000 7f937a80a700 1 Processor -- start 2024-05-03T18:46:16.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:16.617+0000 7f937a80a700 1 -- start start 2024-05-03T18:46:16.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:16.618+0000 7f937a80a700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f93741023d0 0x7f937419e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:16.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:16.618+0000 7f937a80a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9374102d90 0x7f937419ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:16.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:16.618+0000 7f937a80a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9374108440 0x7f93741988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:16.620 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:16.618+0000 7f937a80a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f93741125b0 con 0x7f9374108440 2024-05-03T18:46:16.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:16.618+0000 7f937a80a700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f9374112430 con 0x7f9374102d90 2024-05-03T18:46:16.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:16.618+0000 7f937a80a700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f9374112730 con 0x7f93741023d0 2024-05-03T18:46:16.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:16.618+0000 7f9378da7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9374108440 0x7f93741988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:16.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:16.618+0000 7f9378da7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9374108440 0x7f93741988f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37190/0 (socket says 172.21.15.32:37190) 2024-05-03T18:46:16.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:16.618+0000 7f9378da7700 1 -- 172.21.15.32:0/1571062528 learned_addr learned my addr 172.21.15.32:0/1571062528 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:46:16.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:16.618+0000 7f93737fe700 1 --2- 172.21.15.32:0/1571062528 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9374102d90 0x7f937419ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:16.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:16.618+0000 7f9373fff700 1 --2- 172.21.15.32:0/1571062528 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f93741023d0 0x7f937419e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:16.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:16.618+0000 7f9378da7700 1 -- 172.21.15.32:0/1571062528 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f93741023d0 msgr2=0x7f937419e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:16.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:16.618+0000 7f9378da7700 1 --2- 172.21.15.32:0/1571062528 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f93741023d0 0x7f937419e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:16.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:16.618+0000 7f9378da7700 1 -- 172.21.15.32:0/1571062528 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9374102d90 msgr2=0x7f937419ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:16.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:16.618+0000 7f9378da7700 1 --2- 172.21.15.32:0/1571062528 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9374102d90 0x7f937419ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:16.621 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:16.618+0000 7f9378da7700 1 -- 172.21.15.32:0/1571062528 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9374199050 con 0x7f9374108440 2024-05-03T18:46:16.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:16.618+0000 7f9378da7700 1 --2- 172.21.15.32:0/1571062528 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9374108440 0x7f93741988f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f936c007ee0 tx=0x7f936c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:46:16.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:16.619+0000 7f93717fa700 1 -- 172.21.15.32:0/1571062528 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f936c00dbd0 con 0x7f9374108440 2024-05-03T18:46:16.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:16.619+0000 7f93717fa700 1 -- 172.21.15.32:0/1571062528 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f936c00dd70 con 0x7f9374108440 2024-05-03T18:46:16.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:16.619+0000 7f937a80a700 1 -- 172.21.15.32:0/1571062528 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9374199340 con 0x7f9374108440 2024-05-03T18:46:16.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:16.619+0000 7f93717fa700 1 -- 172.21.15.32:0/1571062528 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f936c005990 con 0x7f9374108440 2024-05-03T18:46:16.622 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:16.619+0000 7f937a80a700 1 -- 172.21.15.32:0/1571062528 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93741a5600 con 0x7f9374108440 2024-05-03T18:46:16.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:16.620+0000 7f937a80a700 1 -- 172.21.15.32:0/1571062528 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f937406a190 con 0x7f9374108440 2024-05-03T18:46:16.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:16.621+0000 7f93717fa700 1 -- 172.21.15.32:0/1571062528 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f936c01a020 con 0x7f9374108440 2024-05-03T18:46:16.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:16.622+0000 7f93717fa700 1 --2- 172.21.15.32:0/1571062528 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f93600b2910 0x7f93600b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:16.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:16.622+0000 7f9373fff700 1 --2- 172.21.15.32:0/1571062528 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f93600b2910 0x7f93600b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:16.623 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:16.622+0000 7f93717fa700 1 -- 172.21.15.32:0/1571062528 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8269+0+0 (secure 0 0 0) 0x7f936c0e1c60 con 0x7f9374108440 2024-05-03T18:46:16.624 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:16.623+0000 7f9373fff700 1 --2- 172.21.15.32:0/1571062528 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f93600b2910 0x7f93600b4dd0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f9374199a30 tx=0x7f936400b5b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:46:16.628 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:16.627+0000 7f93717fa700 1 -- 172.21.15.32:0/1571062528 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f936c010070 con 0x7f9374108440 2024-05-03T18:46:16.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:16.845+0000 7f937a80a700 1 -- 172.21.15.32:0/1571062528 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f937410a340 con 0x7f9374108440 2024-05-03T18:46:16.846 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:16.845+0000 7f93717fa700 1 -- 172.21.15.32:0/1571062528 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f936c0a9800 con 0x7f9374108440 2024-05-03T18:46:16.846 INFO:teuthology.orchestra.run.smithi032.stdout:55834574930 2024-05-03T18:46:16.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:16.847+0000 7f937a80a700 1 -- 172.21.15.32:0/1571062528 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f93600b2910 msgr2=0x7f93600b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:16.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:16.847+0000 7f937a80a700 1 --2- 172.21.15.32:0/1571062528 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f93600b2910 0x7f93600b4dd0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f9374199a30 tx=0x7f936400b5b0 comp rx=0 tx=0).stop 2024-05-03T18:46:16.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:16.847+0000 7f937a80a700 1 -- 172.21.15.32:0/1571062528 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9374108440 msgr2=0x7f93741988f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:16.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:16.847+0000 7f937a80a700 1 --2- 172.21.15.32:0/1571062528 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9374108440 0x7f93741988f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f936c007ee0 tx=0x7f936c00d3b0 comp rx=0 tx=0).stop 2024-05-03T18:46:16.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:16.847+0000 7f937a80a700 1 -- 172.21.15.32:0/1571062528 shutdown_connections 2024-05-03T18:46:16.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:16.847+0000 7f937a80a700 1 --2- 172.21.15.32:0/1571062528 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f93600b2910 0x7f93600b4dd0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:16.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:16.847+0000 7f937a80a700 1 --2- 172.21.15.32:0/1571062528 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9374108440 0x7f93741988f0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:16.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:16.847+0000 7f937a80a700 1 --2- 172.21.15.32:0/1571062528 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f93741023d0 0x7f937419e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:16.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:16.847+0000 7f937a80a700 1 --2- 172.21.15.32:0/1571062528 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9374102d90 0x7f937419ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:16.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:16.847+0000 7f937a80a700 1 -- 172.21.15.32:0/1571062528 >> 172.21.15.32:0/1571062528 conn(0x7f93740793e0 msgr2=0x7f9374101200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:46:16.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:16.847+0000 7f937a80a700 1 -- 172.21.15.32:0/1571062528 shutdown_connections 2024-05-03T18:46:16.849 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:16.847+0000 7f937a80a700 1 -- 172.21.15.32:0/1571062528 wait complete. 2024-05-03T18:46:17.262 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:46:17 smithi032 ceph-mon[29763]: pgmap v325: 97 pgs: 97 active+clean; 631 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:46:17.262 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:46:17 smithi032 ceph-mon[29763]: from='client.? 172.21.15.32:0/1571062528' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-03T18:46:17.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:46:17 smithi045 ceph-mon[33299]: pgmap v325: 97 pgs: 97 active+clean; 631 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:46:17.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:46:17 smithi045 ceph-mon[33299]: from='client.? 172.21.15.32:0/1571062528' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-03T18:46:17.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:46:17 smithi113 ceph-mon[33540]: pgmap v325: 97 pgs: 97 active+clean; 631 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:46:17.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:46:17 smithi113 ceph-mon[33540]: from='client.? 172.21.15.32:0/1571062528' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-03T18:46:17.592 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574925 got 55834574930 for osd.1 2024-05-03T18:46:17.593 DEBUG:teuthology.parallel:result is None 2024-05-03T18:46:17.767 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:46:17.767 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:46:18.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:18.034+0000 7efd2c84a700 1 -- 172.21.15.32:0/2171972299 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efd28102fd0 msgr2=0x7efd28107020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:18.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:18.034+0000 7efd2c84a700 1 --2- 172.21.15.32:0/2171972299 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efd28102fd0 0x7efd28107020 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7efd1c0099c0 tx=0x7efd1c009cd0 comp rx=0 tx=0).stop 2024-05-03T18:46:18.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:18.034+0000 7efd2c84a700 1 -- 172.21.15.32:0/2171972299 shutdown_connections 2024-05-03T18:46:18.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:18.034+0000 7efd2c84a700 1 --2- 172.21.15.32:0/2171972299 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd28107700 0x7efd2810f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:18.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:18.034+0000 7efd2c84a700 1 --2- 172.21.15.32:0/2171972299 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efd28102620 0x7efd28102a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:18.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:18.034+0000 7efd2c84a700 1 --2- 172.21.15.32:0/2171972299 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efd28102fd0 0x7efd28107020 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:18.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:18.034+0000 7efd2c84a700 1 -- 172.21.15.32:0/2171972299 >> 172.21.15.32:0/2171972299 conn(0x7efd28079440 msgr2=0x7efd28079850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:46:18.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:18.034+0000 7efd2c84a700 1 -- 172.21.15.32:0/2171972299 shutdown_connections 2024-05-03T18:46:18.036 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:18.034+0000 7efd2c84a700 1 -- 172.21.15.32:0/2171972299 wait complete. 2024-05-03T18:46:18.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:18.035+0000 7efd2c84a700 1 Processor -- start 2024-05-03T18:46:18.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:18.035+0000 7efd2c84a700 1 -- start start 2024-05-03T18:46:18.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:18.035+0000 7efd2c84a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efd28102620 0x7efd2819aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:18.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:18.035+0000 7efd2c84a700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd28102fd0 0x7efd2819afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:18.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:18.035+0000 7efd2c84a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efd28107700 0x7efd281a5af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:18.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:18.035+0000 7efd2c84a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7efd281125f0 con 0x7efd28102620 2024-05-03T18:46:18.038 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:18.035+0000 7efd2c84a700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7efd28112470 con 0x7efd28107700 2024-05-03T18:46:18.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:18.035+0000 7efd2c84a700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7efd28112770 con 0x7efd28102fd0 2024-05-03T18:46:18.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:18.035+0000 7efd2659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efd28102620 0x7efd2819aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:18.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:18.035+0000 7efd2659c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efd28102620 0x7efd2819aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37204/0 (socket says 172.21.15.32:37204) 2024-05-03T18:46:18.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:18.035+0000 7efd2659c700 1 -- 172.21.15.32:0/2180225626 learned_addr learned my addr 172.21.15.32:0/2180225626 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:46:18.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:18.035+0000 7efd26d9d700 1 --2- 172.21.15.32:0/2180225626 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efd28107700 0x7efd281a5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:18.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:18.036+0000 7efd25d9b700 1 --2- 172.21.15.32:0/2180225626 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd28102fd0 0x7efd2819afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:18.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:18.036+0000 7efd2659c700 1 -- 172.21.15.32:0/2180225626 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd28102fd0 msgr2=0x7efd2819afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:18.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:18.036+0000 7efd2659c700 1 --2- 172.21.15.32:0/2180225626 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efd28102fd0 0x7efd2819afb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:18.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:18.036+0000 7efd2659c700 1 -- 172.21.15.32:0/2180225626 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efd28107700 msgr2=0x7efd281a5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:18.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:18.036+0000 7efd2659c700 1 --2- 172.21.15.32:0/2180225626 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efd28107700 0x7efd281a5af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:18.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:18.036+0000 7efd2659c700 1 -- 172.21.15.32:0/2180225626 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd281a6030 con 0x7efd28102620 2024-05-03T18:46:18.039 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:18.036+0000 7efd26d9d700 1 --2- 172.21.15.32:0/2180225626 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efd28107700 0x7efd281a5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T18:46:18.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:18.036+0000 7efd2659c700 1 --2- 172.21.15.32:0/2180225626 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efd28102620 0x7efd2819aa70 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7efd1800d980 tx=0x7efd1800dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:46:18.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:18.036+0000 7efd177fe700 1 -- 172.21.15.32:0/2180225626 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd18016070 con 0x7efd28102620 2024-05-03T18:46:18.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:18.036+0000 7efd177fe700 1 -- 172.21.15.32:0/2180225626 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd180044d0 con 0x7efd28102620 2024-05-03T18:46:18.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:18.036+0000 7efd2c84a700 1 -- 172.21.15.32:0/2180225626 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd281a6260 con 0x7efd28102620 2024-05-03T18:46:18.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:18.036+0000 7efd177fe700 1 -- 172.21.15.32:0/2180225626 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd18004d00 con 0x7efd28102620 2024-05-03T18:46:18.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:18.036+0000 7efd2c84a700 1 -- 172.21.15.32:0/2180225626 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd281a6770 con 0x7efd28102620 2024-05-03T18:46:18.040 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:18.037+0000 7efd2c84a700 1 -- 172.21.15.32:0/2180225626 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7efd28063b70 con 0x7efd28102620 2024-05-03T18:46:18.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:18.039+0000 7efd177fe700 1 -- 172.21.15.32:0/2180225626 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7efd18004020 con 0x7efd28102620 2024-05-03T18:46:18.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:18.039+0000 7efd177fe700 1 --2- 172.21.15.32:0/2180225626 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7efd100b2910 0x7efd100b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:18.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:18.039+0000 7efd25d9b700 1 --2- 172.21.15.32:0/2180225626 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7efd100b2910 0x7efd100b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:18.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:18.039+0000 7efd177fe700 1 -- 172.21.15.32:0/2180225626 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8269+0+0 (secure 0 0 0) 0x7efd180e2960 con 0x7efd28102620 2024-05-03T18:46:18.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:18.040+0000 7efd177fe700 1 --2- 172.21.15.32:0/2180225626 >> [v2:172.21.15.32:6810/2606514063,v1:172.21.15.32:6811/2606514063] conn(0x7efd100bc550 0x7efd100be9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:18.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:18.040+0000 7efd177fe700 1 -- 172.21.15.32:0/2180225626 --> [v2:172.21.15.32:6810/2606514063,v1:172.21.15.32:6811/2606514063] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7efd1801be50 con 0x7efd100bc550 2024-05-03T18:46:18.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:18.040+0000 7efd177fe700 1 -- 172.21.15.32:0/2180225626 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7efd180e2db0 con 0x7efd28102620 2024-05-03T18:46:18.041 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:18.040+0000 7efd26d9d700 1 --2- 172.21.15.32:0/2180225626 >> [v2:172.21.15.32:6810/2606514063,v1:172.21.15.32:6811/2606514063] conn(0x7efd100bc550 0x7efd100be9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:18.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:18.040+0000 7efd25d9b700 1 --2- 172.21.15.32:0/2180225626 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7efd100b2910 0x7efd100b4dd0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7efd1c00b580 tx=0x7efd1c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:46:18.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:18.040+0000 7efd26d9d700 1 --2- 172.21.15.32:0/2180225626 >> [v2:172.21.15.32:6810/2606514063,v1:172.21.15.32:6811/2606514063] conn(0x7efd100bc550 0x7efd100be9f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:46:18.042 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:18.042+0000 7efd177fe700 1 -- 172.21.15.32:0/2180225626 <== osd.3 v2:172.21.15.32:6810/2606514063 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7efd1801be50 con 0x7efd100bc550 2024-05-03T18:46:18.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:18.065+0000 7efd2c84a700 1 -- 172.21.15.32:0/2180225626 --> [v2:172.21.15.32:6810/2606514063,v1:172.21.15.32:6811/2606514063] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7efd28064020 con 0x7efd100bc550 2024-05-03T18:46:18.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:18.066+0000 7efd177fe700 1 -- 172.21.15.32:0/2180225626 <== osd.3 v2:172.21.15.32:6810/2606514063 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7efd28064020 con 0x7efd100bc550 2024-05-03T18:46:18.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:18.066+0000 7efd2c84a700 1 -- 172.21.15.32:0/2180225626 >> [v2:172.21.15.32:6810/2606514063,v1:172.21.15.32:6811/2606514063] conn(0x7efd100bc550 msgr2=0x7efd100be9f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:18.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:18.066+0000 7efd2c84a700 1 --2- 172.21.15.32:0/2180225626 >> [v2:172.21.15.32:6810/2606514063,v1:172.21.15.32:6811/2606514063] conn(0x7efd100bc550 0x7efd100be9f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:18.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:18.066+0000 7efd2c84a700 1 -- 172.21.15.32:0/2180225626 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7efd100b2910 msgr2=0x7efd100b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:18.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:18.066+0000 7efd2c84a700 1 --2- 172.21.15.32:0/2180225626 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7efd100b2910 0x7efd100b4dd0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7efd1c00b580 tx=0x7efd1c000bc0 comp rx=0 tx=0).stop 2024-05-03T18:46:18.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:18.066+0000 7efd2c84a700 1 -- 172.21.15.32:0/2180225626 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efd28102620 msgr2=0x7efd2819aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:18.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:18.066+0000 7efd2c84a700 1 --2- 172.21.15.32:0/2180225626 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efd28102620 0x7efd2819aa70 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7efd1800d980 tx=0x7efd1800dd40 comp rx=0 tx=0).stop 2024-05-03T18:46:18.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:18.066+0000 7efd26d9d700 1 -- 172.21.15.32:0/2180225626 reap_dead start 2024-05-03T18:46:18.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:18.066+0000 7efd2c84a700 1 -- 172.21.15.32:0/2180225626 shutdown_connections 2024-05-03T18:46:18.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:18.066+0000 7efd2c84a700 1 -- 172.21.15.32:0/2180225626 >> 172.21.15.32:0/2180225626 conn(0x7efd28079440 msgr2=0x7efd28100370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:46:18.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:18.066+0000 7efd2c84a700 1 -- 172.21.15.32:0/2180225626 shutdown_connections 2024-05-03T18:46:18.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:18.066+0000 7efd2c84a700 1 -- 172.21.15.32:0/2180225626 wait complete. 2024-05-03T18:46:19.218 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:46:19 smithi032 ceph-mon[29763]: pgmap v326: 97 pgs: 97 active+clean; 631 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:46:19.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:46:19 smithi045 ceph-mon[33299]: pgmap v326: 97 pgs: 97 active+clean; 631 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:46:19.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:46:19 smithi113 ceph-mon[33540]: pgmap v326: 97 pgs: 97 active+clean; 631 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:46:19.974 INFO:teuthology.orchestra.run.smithi032.stdout:107374182472 2024-05-03T18:46:19.975 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid f500455e-097b-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-05-03T18:46:20.694 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:46:20.694 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:46:20.695 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:46:21.071 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:46:21.317 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:46:21 smithi032 ceph-mon[29763]: pgmap v327: 97 pgs: 97 active+clean; 631 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:46:21.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:46:21 smithi045 ceph-mon[33299]: pgmap v327: 97 pgs: 97 active+clean; 631 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:46:21.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:46:21 smithi113 ceph-mon[33540]: pgmap v327: 97 pgs: 97 active+clean; 631 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:46:22.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:46:22 smithi032 ceph-mon[29763]: from='client.? 172.21.15.32:0/286429739' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-03T18:46:22.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:46:22 smithi045 ceph-mon[33299]: from='client.? 172.21.15.32:0/286429739' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-03T18:46:22.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:46:22 smithi113 ceph-mon[33540]: from='client.? 172.21.15.32:0/286429739' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-03T18:46:22.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:20.901+0000 7f3e02ac6700 1 -- 172.21.15.32:0/1440876433 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3dfc10fea0 msgr2=0x7f3dfc112290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:22.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:20.901+0000 7f3e02ac6700 1 --2- 172.21.15.32:0/1440876433 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3dfc10fea0 0x7f3dfc112290 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f3df400b0a0 tx=0x7f3df400b3b0 comp rx=0 tx=0).stop 2024-05-03T18:46:22.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:20.901+0000 7f3e02ac6700 1 -- 172.21.15.32:0/1440876433 shutdown_connections 2024-05-03T18:46:22.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:20.901+0000 7f3e02ac6700 1 --2- 172.21.15.32:0/1440876433 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3dfc10fea0 0x7f3dfc112290 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:22.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:20.901+0000 7f3e02ac6700 1 --2- 172.21.15.32:0/1440876433 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3dfc0756d0 0x7f3dfc075ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:22.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:20.901+0000 7f3e02ac6700 1 --2- 172.21.15.32:0/1440876433 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3dfc075ff0 0x7f3dfc10f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:22.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:20.901+0000 7f3e02ac6700 1 -- 172.21.15.32:0/1440876433 >> 172.21.15.32:0/1440876433 conn(0x7f3dfc07a600 msgr2=0x7f3dfc07aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:46:22.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:20.901+0000 7f3e02ac6700 1 -- 172.21.15.32:0/1440876433 shutdown_connections 2024-05-03T18:46:22.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:20.901+0000 7f3e02ac6700 1 -- 172.21.15.32:0/1440876433 wait complete. 2024-05-03T18:46:22.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:20.902+0000 7f3e02ac6700 1 Processor -- start 2024-05-03T18:46:22.932 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:20.902+0000 7f3e02ac6700 1 -- start start 2024-05-03T18:46:22.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:20.902+0000 7f3e02ac6700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3dfc0756d0 0x7f3dfc1a1a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:22.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:20.902+0000 7f3e02ac6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3dfc075ff0 0x7f3dfc1a1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:22.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:20.902+0000 7f3e02ac6700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3dfc10fea0 0x7f3dfc19bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:22.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:20.902+0000 7f3e02ac6700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f3dfc115920 con 0x7f3dfc075ff0 2024-05-03T18:46:22.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:20.902+0000 7f3e02ac6700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3dfc1157a0 con 0x7f3dfc10fea0 2024-05-03T18:46:22.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:20.902+0000 7f3e02ac6700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f3dfc115aa0 con 0x7f3dfc0756d0 2024-05-03T18:46:22.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:20.902+0000 7f3dfbfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3dfc075ff0 0x7f3dfc1a1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:22.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:20.902+0000 7f3dfbfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3dfc075ff0 0x7f3dfc1a1fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37220/0 (socket says 172.21.15.32:37220) 2024-05-03T18:46:22.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:20.902+0000 7f3dfbfff700 1 -- 172.21.15.32:0/286429739 learned_addr learned my addr 172.21.15.32:0/286429739 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:46:22.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:20.902+0000 7f3e01063700 1 --2- 172.21.15.32:0/286429739 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3dfc10fea0 0x7f3dfc19bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:22.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:20.902+0000 7f3e00862700 1 --2- 172.21.15.32:0/286429739 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3dfc0756d0 0x7f3dfc1a1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:22.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:20.902+0000 7f3dfbfff700 1 -- 172.21.15.32:0/286429739 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3dfc0756d0 msgr2=0x7f3dfc1a1a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:22.933 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:20.902+0000 7f3dfbfff700 1 --2- 172.21.15.32:0/286429739 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3dfc0756d0 0x7f3dfc1a1a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:22.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:20.902+0000 7f3dfbfff700 1 -- 172.21.15.32:0/286429739 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3dfc10fea0 msgr2=0x7f3dfc19bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:22.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:20.902+0000 7f3dfbfff700 1 --2- 172.21.15.32:0/286429739 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3dfc10fea0 0x7f3dfc19bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:22.934 INFO:teuthology.orchestra.run.smithi032.stdout:180388626488 2024-05-03T18:46:22.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:20.903+0000 7f3dfbfff700 1 -- 172.21.15.32:0/286429739 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3dfc19c460 con 0x7f3dfc075ff0 2024-05-03T18:46:22.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:20.903+0000 7f3e01063700 1 --2- 172.21.15.32:0/286429739 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3dfc10fea0 0x7f3dfc19bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T18:46:22.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:20.903+0000 7f3dfbfff700 1 --2- 172.21.15.32:0/286429739 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3dfc075ff0 0x7f3dfc1a1fa0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f3df000b500 tx=0x7f3df000b8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:46:22.934 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:20.903+0000 7f3df9ffb700 1 -- 172.21.15.32:0/286429739 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3df0015020 con 0x7f3dfc075ff0 2024-05-03T18:46:22.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:20.903+0000 7f3e02ac6700 1 -- 172.21.15.32:0/286429739 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3dfc19c750 con 0x7f3dfc075ff0 2024-05-03T18:46:22.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:20.903+0000 7f3df9ffb700 1 -- 172.21.15.32:0/286429739 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3df0011ce0 con 0x7f3dfc075ff0 2024-05-03T18:46:22.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:20.903+0000 7f3df9ffb700 1 -- 172.21.15.32:0/286429739 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3df0010d60 con 0x7f3dfc075ff0 2024-05-03T18:46:22.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:20.903+0000 7f3e02ac6700 1 -- 172.21.15.32:0/286429739 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3dfc1a89e0 con 0x7f3dfc075ff0 2024-05-03T18:46:22.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:20.904+0000 7f3e02ac6700 1 -- 172.21.15.32:0/286429739 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3dfc076dc0 con 0x7f3dfc075ff0 2024-05-03T18:46:22.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:20.905+0000 7f3df9ffb700 1 -- 172.21.15.32:0/286429739 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f3df0018020 con 0x7f3dfc075ff0 2024-05-03T18:46:22.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:20.906+0000 7f3df9ffb700 1 --2- 172.21.15.32:0/286429739 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f3de40b2910 0x7f3de40b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:22.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:20.906+0000 7f3e00862700 1 --2- 172.21.15.32:0/286429739 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f3de40b2910 0x7f3de40b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:22.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:20.906+0000 7f3df9ffb700 1 -- 172.21.15.32:0/286429739 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8269+0+0 (secure 0 0 0) 0x7f3df00e1b30 con 0x7f3dfc075ff0 2024-05-03T18:46:22.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:20.906+0000 7f3e00862700 1 --2- 172.21.15.32:0/286429739 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f3de40b2910 0x7f3de40b4dd0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f3dec00ba60 tx=0x7f3dec005d50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:46:22.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:20.910+0000 7f3df9ffb700 1 -- 172.21.15.32:0/286429739 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f3df001f020 con 0x7f3dfc075ff0 2024-05-03T18:46:22.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:21.136+0000 7f3e02ac6700 1 -- 172.21.15.32:0/286429739 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f3dfc1a8bb0 con 0x7f3dfc075ff0 2024-05-03T18:46:22.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:21.137+0000 7f3df9ffb700 1 -- 172.21.15.32:0/286429739 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f3df00a9780 con 0x7f3dfc075ff0 2024-05-03T18:46:22.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:21.139+0000 7f3e02ac6700 1 -- 172.21.15.32:0/286429739 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f3de40b2910 msgr2=0x7f3de40b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:22.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:21.139+0000 7f3e02ac6700 1 --2- 172.21.15.32:0/286429739 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f3de40b2910 0x7f3de40b4dd0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f3dec00ba60 tx=0x7f3dec005d50 comp rx=0 tx=0).stop 2024-05-03T18:46:22.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:21.139+0000 7f3e02ac6700 1 -- 172.21.15.32:0/286429739 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3dfc075ff0 msgr2=0x7f3dfc1a1fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:22.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:21.139+0000 7f3e02ac6700 1 --2- 172.21.15.32:0/286429739 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3dfc075ff0 0x7f3dfc1a1fa0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f3df000b500 tx=0x7f3df000b8c0 comp rx=0 tx=0).stop 2024-05-03T18:46:22.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:21.139+0000 7f3e02ac6700 1 -- 172.21.15.32:0/286429739 shutdown_connections 2024-05-03T18:46:22.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:21.139+0000 7f3e02ac6700 1 --2- 172.21.15.32:0/286429739 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f3de40b2910 0x7f3de40b4dd0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:22.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:21.139+0000 7f3e02ac6700 1 --2- 172.21.15.32:0/286429739 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3dfc10fea0 0x7f3dfc19bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:22.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:21.139+0000 7f3e02ac6700 1 --2- 172.21.15.32:0/286429739 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3dfc0756d0 0x7f3dfc1a1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:22.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:21.139+0000 7f3e02ac6700 1 --2- 172.21.15.32:0/286429739 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3dfc075ff0 0x7f3dfc1a1fa0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:22.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:21.139+0000 7f3e02ac6700 1 -- 172.21.15.32:0/286429739 >> 172.21.15.32:0/286429739 conn(0x7f3dfc07a600 msgr2=0x7f3dfc100610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:46:22.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:21.139+0000 7f3e02ac6700 1 -- 172.21.15.32:0/286429739 shutdown_connections 2024-05-03T18:46:22.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:21.139+0000 7f3e02ac6700 1 -- 172.21.15.32:0/286429739 wait complete. 2024-05-03T18:46:23.261 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:46:23 smithi032 ceph-mon[29763]: pgmap v328: 97 pgs: 97 active+clean; 631 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:46:23.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:46:23 smithi045 ceph-mon[33299]: pgmap v328: 97 pgs: 97 active+clean; 631 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:46:23.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:46:23 smithi113 ceph-mon[33540]: pgmap v328: 97 pgs: 97 active+clean; 631 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:46:23.577 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:46:23.592 INFO:tasks.cephadm.ceph_manager.ceph:need seq 180388626484 got 180388626488 for osd.6 2024-05-03T18:46:23.593 DEBUG:teuthology.parallel:result is None 2024-05-03T18:46:24.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:24.062+0000 7f74df836700 1 -- 172.21.15.32:0/3719793648 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74d806b180 msgr2=0x7f74d806b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:24.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:24.062+0000 7f74df836700 1 --2- 172.21.15.32:0/3719793648 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74d806b180 0x7f74d806b600 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f74c8009990 tx=0x7f74c8009ca0 comp rx=0 tx=0).stop 2024-05-03T18:46:24.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:24.062+0000 7f74df836700 1 -- 172.21.15.32:0/3719793648 shutdown_connections 2024-05-03T18:46:24.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:24.062+0000 7f74df836700 1 --2- 172.21.15.32:0/3719793648 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f74d806bb40 0x7f74d8110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:24.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:24.062+0000 7f74df836700 1 --2- 172.21.15.32:0/3719793648 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f74d8109510 0x7f74d81098f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:24.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:24.062+0000 7f74df836700 1 --2- 172.21.15.32:0/3719793648 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74d806b180 0x7f74d806b600 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:24.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:24.062+0000 7f74df836700 1 -- 172.21.15.32:0/3719793648 >> 172.21.15.32:0/3719793648 conn(0x7f74d8079350 msgr2=0x7f74d8079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:46:24.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:24.062+0000 7f74df836700 1 -- 172.21.15.32:0/3719793648 shutdown_connections 2024-05-03T18:46:24.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:24.062+0000 7f74df836700 1 -- 172.21.15.32:0/3719793648 wait complete. 2024-05-03T18:46:24.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:24.063+0000 7f74df836700 1 Processor -- start 2024-05-03T18:46:24.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:24.063+0000 7f74df836700 1 -- start start 2024-05-03T18:46:24.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:24.063+0000 7f74df836700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74d806b180 0x7f74d819f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:24.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:24.063+0000 7f74df836700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f74d806bb40 0x7f74d819fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:24.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:24.063+0000 7f74df836700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f74d8109510 0x7f74d8199b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:24.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:24.063+0000 7f74df836700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f74d8113720 con 0x7f74d806b180 2024-05-03T18:46:24.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:24.063+0000 7f74df836700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f74d81135a0 con 0x7f74d8109510 2024-05-03T18:46:24.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:24.063+0000 7f74df836700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f74d81138a0 con 0x7f74d806bb40 2024-05-03T18:46:24.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:24.063+0000 7f74dd5d2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74d806b180 0x7f74d819f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:24.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:24.063+0000 7f74dd5d2700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74d806b180 0x7f74d819f940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51492/0 (socket says 172.21.15.32:51492) 2024-05-03T18:46:24.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:24.064+0000 7f74dd5d2700 1 -- 172.21.15.32:0/3927348482 learned_addr learned my addr 172.21.15.32:0/3927348482 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:46:24.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:24.064+0000 7f74dddd3700 1 --2- 172.21.15.32:0/3927348482 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f74d8109510 0x7f74d8199b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:24.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:24.064+0000 7f74dcdd1700 1 --2- 172.21.15.32:0/3927348482 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f74d806bb40 0x7f74d819fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:24.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:24.064+0000 7f74dd5d2700 1 -- 172.21.15.32:0/3927348482 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f74d806bb40 msgr2=0x7f74d819fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:24.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:24.064+0000 7f74dd5d2700 1 --2- 172.21.15.32:0/3927348482 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f74d806bb40 0x7f74d819fe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:24.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:24.064+0000 7f74dd5d2700 1 -- 172.21.15.32:0/3927348482 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f74d8109510 msgr2=0x7f74d8199b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:24.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:24.064+0000 7f74dd5d2700 1 --2- 172.21.15.32:0/3927348482 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f74d8109510 0x7f74d8199b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:24.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:24.064+0000 7f74dd5d2700 1 -- 172.21.15.32:0/3927348482 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74d819a260 con 0x7f74d806b180 2024-05-03T18:46:24.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:24.064+0000 7f74dd5d2700 1 --2- 172.21.15.32:0/3927348482 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74d806b180 0x7f74d819f940 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f74d400b530 tx=0x7f74d400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:46:24.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:24.064+0000 7f74ce7fc700 1 -- 172.21.15.32:0/3927348482 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74d4015020 con 0x7f74d806b180 2024-05-03T18:46:24.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:24.064+0000 7f74df836700 1 -- 172.21.15.32:0/3927348482 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74d819a550 con 0x7f74d806b180 2024-05-03T18:46:24.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:24.064+0000 7f74ce7fc700 1 -- 172.21.15.32:0/3927348482 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74d4011ce0 con 0x7f74d806b180 2024-05-03T18:46:24.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:24.064+0000 7f74ce7fc700 1 -- 172.21.15.32:0/3927348482 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74d4010dd0 con 0x7f74d806b180 2024-05-03T18:46:24.068 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:24.064+0000 7f74df836700 1 -- 172.21.15.32:0/3927348482 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74d81a6810 con 0x7f74d806b180 2024-05-03T18:46:24.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:24.066+0000 7f74df836700 1 -- 172.21.15.32:0/3927348482 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74d810b9f0 con 0x7f74d806b180 2024-05-03T18:46:24.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:24.067+0000 7f74ce7fc700 1 -- 172.21.15.32:0/3927348482 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f74d4018020 con 0x7f74d806b180 2024-05-03T18:46:24.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:24.068+0000 7f74ce7fc700 1 --2- 172.21.15.32:0/3927348482 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f74c40b2910 0x7f74c40b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:24.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:24.068+0000 7f74dcdd1700 1 --2- 172.21.15.32:0/3927348482 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f74c40b2910 0x7f74c40b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:24.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:24.068+0000 7f74ce7fc700 1 -- 172.21.15.32:0/3927348482 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8269+0+0 (secure 0 0 0) 0x7f74d40e1980 con 0x7f74d806b180 2024-05-03T18:46:24.069 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:24.068+0000 7f74dcdd1700 1 --2- 172.21.15.32:0/3927348482 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f74c40b2910 0x7f74c40b4dd0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f74c80098e0 tx=0x7f74c8005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:46:24.073 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:24.072+0000 7f74ce7fc700 1 -- 172.21.15.32:0/3927348482 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f74d401f020 con 0x7f74d806b180 2024-05-03T18:46:24.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:24.289+0000 7f74df836700 1 -- 172.21.15.32:0/3927348482 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f74d819aff0 con 0x7f74d806b180 2024-05-03T18:46:24.290 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:24.289+0000 7f74ce7fc700 1 -- 172.21.15.32:0/3927348482 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f74d40a9630 con 0x7f74d806b180 2024-05-03T18:46:24.291 INFO:teuthology.orchestra.run.smithi032.stdout:133143986245 2024-05-03T18:46:24.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:24.291+0000 7f74df836700 1 -- 172.21.15.32:0/3927348482 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f74c40b2910 msgr2=0x7f74c40b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:24.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:24.291+0000 7f74df836700 1 --2- 172.21.15.32:0/3927348482 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f74c40b2910 0x7f74c40b4dd0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f74c80098e0 tx=0x7f74c8005960 comp rx=0 tx=0).stop 2024-05-03T18:46:24.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:24.291+0000 7f74df836700 1 -- 172.21.15.32:0/3927348482 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74d806b180 msgr2=0x7f74d819f940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:24.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:24.291+0000 7f74df836700 1 --2- 172.21.15.32:0/3927348482 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74d806b180 0x7f74d819f940 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f74d400b530 tx=0x7f74d400b840 comp rx=0 tx=0).stop 2024-05-03T18:46:24.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:24.292+0000 7f74df836700 1 -- 172.21.15.32:0/3927348482 shutdown_connections 2024-05-03T18:46:24.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:24.292+0000 7f74df836700 1 --2- 172.21.15.32:0/3927348482 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f74c40b2910 0x7f74c40b4dd0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:24.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:24.292+0000 7f74df836700 1 --2- 172.21.15.32:0/3927348482 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f74d8109510 0x7f74d8199b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:24.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:24.292+0000 7f74df836700 1 --2- 172.21.15.32:0/3927348482 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74d806b180 0x7f74d819f940 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:24.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:24.292+0000 7f74df836700 1 --2- 172.21.15.32:0/3927348482 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f74d806bb40 0x7f74d819fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:24.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:24.292+0000 7f74df836700 1 -- 172.21.15.32:0/3927348482 >> 172.21.15.32:0/3927348482 conn(0x7f74d8079350 msgr2=0x7f74d8101220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:46:24.293 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:24.292+0000 7f74df836700 1 -- 172.21.15.32:0/3927348482 shutdown_connections 2024-05-03T18:46:24.294 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:24.292+0000 7f74df836700 1 -- 172.21.15.32:0/3927348482 wait complete. 2024-05-03T18:46:24.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:24.909+0000 7f17b0ee1700 1 -- 172.21.15.32:0/1927234970 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f17ac1023a0 msgr2=0x7f17ac102820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:24.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:24.909+0000 7f17b0ee1700 1 --2- 172.21.15.32:0/1927234970 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f17ac1023a0 0x7f17ac102820 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f17a0009990 tx=0x7f17a0009ca0 comp rx=0 tx=0).stop 2024-05-03T18:46:24.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:24.909+0000 7f17b0ee1700 1 -- 172.21.15.32:0/1927234970 shutdown_connections 2024-05-03T18:46:24.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:24.909+0000 7f17b0ee1700 1 --2- 172.21.15.32:0/1927234970 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f17ac102d60 0x7f17ac10f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:24.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:24.909+0000 7f17b0ee1700 1 --2- 172.21.15.32:0/1927234970 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f17ac108410 0x7f17ac1087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:24.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:24.909+0000 7f17b0ee1700 1 --2- 172.21.15.32:0/1927234970 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f17ac1023a0 0x7f17ac102820 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:24.911 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:24.909+0000 7f17b0ee1700 1 -- 172.21.15.32:0/1927234970 >> 172.21.15.32:0/1927234970 conn(0x7f17ac079350 msgr2=0x7f17ac079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:46:24.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:24.909+0000 7f17b0ee1700 1 -- 172.21.15.32:0/1927234970 shutdown_connections 2024-05-03T18:46:24.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:24.909+0000 7f17b0ee1700 1 -- 172.21.15.32:0/1927234970 wait complete. 2024-05-03T18:46:24.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:24.910+0000 7f17b0ee1700 1 Processor -- start 2024-05-03T18:46:24.912 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:24.910+0000 7f17b0ee1700 1 -- start start 2024-05-03T18:46:24.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:24.910+0000 7f17b0ee1700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f17ac1023a0 0x7f17ac104080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:24.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:24.910+0000 7f17b0ee1700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f17ac102d60 0x7f17ac1045c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:24.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:24.910+0000 7f17b0ee1700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f17ac108410 0x7f17ac104b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:24.913 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:24.910+0000 7f17b0ee1700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f17ac112530 con 0x7f17ac102d60 2024-05-03T18:46:24.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:24.910+0000 7f17b0ee1700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f17ac1123b0 con 0x7f17ac108410 2024-05-03T18:46:24.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:24.910+0000 7f17b0ee1700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f17ac1126b0 con 0x7f17ac1023a0 2024-05-03T18:46:24.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:24.911+0000 7f17a9d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f17ac102d60 0x7f17ac1045c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:24.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:24.911+0000 7f17aa59c700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f17ac1023a0 0x7f17ac104080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:24.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:24.911+0000 7f17a9d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f17ac102d60 0x7f17ac1045c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51504/0 (socket says 172.21.15.32:51504) 2024-05-03T18:46:24.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:24.911+0000 7f17a9d9b700 1 -- 172.21.15.32:0/456819420 learned_addr learned my addr 172.21.15.32:0/456819420 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:46:24.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:24.911+0000 7f17a9d9b700 1 -- 172.21.15.32:0/456819420 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f17ac1023a0 msgr2=0x7f17ac104080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:24.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:24.911+0000 7f17a9d9b700 1 --2- 172.21.15.32:0/456819420 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f17ac1023a0 0x7f17ac104080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:24.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:24.911+0000 7f17a9d9b700 1 -- 172.21.15.32:0/456819420 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f17ac108410 msgr2=0x7f17ac104b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:24.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:24.911+0000 7f17aad9d700 1 --2- 172.21.15.32:0/456819420 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f17ac108410 0x7f17ac104b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:24.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:24.911+0000 7f17a9d9b700 1 --2- 172.21.15.32:0/456819420 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f17ac108410 0x7f17ac104b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:24.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:24.911+0000 7f17a9d9b700 1 -- 172.21.15.32:0/456819420 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17ac1a7ff0 con 0x7f17ac102d60 2024-05-03T18:46:24.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:24.911+0000 7f17aad9d700 1 --2- 172.21.15.32:0/456819420 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f17ac108410 0x7f17ac104b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:46:24.914 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:24.911+0000 7f17a9d9b700 1 --2- 172.21.15.32:0/456819420 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f17ac102d60 0x7f17ac1045c0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f17a00057c0 tx=0x7f17a00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:46:24.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:24.911+0000 7f179b7fe700 1 -- 172.21.15.32:0/456819420 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17a0027070 con 0x7f17ac102d60 2024-05-03T18:46:24.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:24.911+0000 7f17b0ee1700 1 -- 172.21.15.32:0/456819420 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17ac1a8280 con 0x7f17ac102d60 2024-05-03T18:46:24.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:24.911+0000 7f179b7fe700 1 -- 172.21.15.32:0/456819420 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17a0003f20 con 0x7f17ac102d60 2024-05-03T18:46:24.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:24.911+0000 7f179b7fe700 1 -- 172.21.15.32:0/456819420 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17a0022bc0 con 0x7f17ac102d60 2024-05-03T18:46:24.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:24.911+0000 7f17b0ee1700 1 -- 172.21.15.32:0/456819420 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17ac1a8760 con 0x7f17ac102d60 2024-05-03T18:46:24.915 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:24.912+0000 7f17b0ee1700 1 -- 172.21.15.32:0/456819420 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17ac06a190 con 0x7f17ac102d60 2024-05-03T18:46:24.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:24.914+0000 7f179b7fe700 1 -- 172.21.15.32:0/456819420 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f17a002a070 con 0x7f17ac102d60 2024-05-03T18:46:24.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:24.915+0000 7f179b7fe700 1 --2- 172.21.15.32:0/456819420 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f17940b2910 0x7f17940b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:24.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:24.915+0000 7f17aa59c700 1 --2- 172.21.15.32:0/456819420 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f17940b2910 0x7f17940b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:24.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:24.915+0000 7f179b7fe700 1 -- 172.21.15.32:0/456819420 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8269+0+0 (secure 0 0 0) 0x7f17a00e3b30 con 0x7f17ac102d60 2024-05-03T18:46:24.916 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:24.915+0000 7f17aa59c700 1 --2- 172.21.15.32:0/456819420 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f17940b2910 0x7f17940b4dd0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f179c00abd0 tx=0x7f179c00a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:46:24.919 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:24.918+0000 7f179b7fe700 1 -- 172.21.15.32:0/456819420 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f17a0031020 con 0x7f17ac102d60 2024-05-03T18:46:25.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:25.135+0000 7f17b0ee1700 1 -- 172.21.15.32:0/456819420 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f17ac1a8a80 con 0x7f17ac102d60 2024-05-03T18:46:25.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:25.135+0000 7f179b7fe700 1 -- 172.21.15.32:0/456819420 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f17a0013070 con 0x7f17ac102d60 2024-05-03T18:46:25.136 INFO:teuthology.orchestra.run.smithi032.stdout:300647710753 2024-05-03T18:46:25.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:25.137+0000 7f17b0ee1700 1 -- 172.21.15.32:0/456819420 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f17940b2910 msgr2=0x7f17940b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:25.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:25.137+0000 7f17b0ee1700 1 --2- 172.21.15.32:0/456819420 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f17940b2910 0x7f17940b4dd0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f179c00abd0 tx=0x7f179c00a380 comp rx=0 tx=0).stop 2024-05-03T18:46:25.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:25.137+0000 7f17b0ee1700 1 -- 172.21.15.32:0/456819420 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f17ac102d60 msgr2=0x7f17ac1045c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:25.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:25.137+0000 7f17b0ee1700 1 --2- 172.21.15.32:0/456819420 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f17ac102d60 0x7f17ac1045c0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f17a00057c0 tx=0x7f17a00057f0 comp rx=0 tx=0).stop 2024-05-03T18:46:25.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:25.138+0000 7f17b0ee1700 1 -- 172.21.15.32:0/456819420 shutdown_connections 2024-05-03T18:46:25.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:25.138+0000 7f17b0ee1700 1 --2- 172.21.15.32:0/456819420 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f17940b2910 0x7f17940b4dd0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:25.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:25.138+0000 7f17b0ee1700 1 --2- 172.21.15.32:0/456819420 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f17ac108410 0x7f17ac104b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:25.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:25.138+0000 7f17b0ee1700 1 --2- 172.21.15.32:0/456819420 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f17ac1023a0 0x7f17ac104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:25.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:25.138+0000 7f17b0ee1700 1 --2- 172.21.15.32:0/456819420 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f17ac102d60 0x7f17ac1045c0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:25.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:25.138+0000 7f17b0ee1700 1 -- 172.21.15.32:0/456819420 >> 172.21.15.32:0/456819420 conn(0x7f17ac079350 msgr2=0x7f17ac101610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:46:25.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:25.138+0000 7f17b0ee1700 1 -- 172.21.15.32:0/456819420 shutdown_connections 2024-05-03T18:46:25.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:25.138+0000 7f17b0ee1700 1 -- 172.21.15.32:0/456819420 wait complete. 2024-05-03T18:46:25.353 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:46:25 smithi113 ceph-mon[33540]: pgmap v329: 97 pgs: 97 active+clean; 631 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:46:25.353 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:46:25 smithi113 ceph-mon[33540]: from='client.? 172.21.15.32:0/3927348482' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-03T18:46:25.357 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:46:25 smithi032 ceph-mon[29763]: pgmap v329: 97 pgs: 97 active+clean; 631 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:46:25.357 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:46:25 smithi032 ceph-mon[29763]: from='client.? 172.21.15.32:0/3927348482' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-03T18:46:25.497 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:46:25 smithi045 ceph-mon[33299]: pgmap v329: 97 pgs: 97 active+clean; 631 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:46:25.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:46:25 smithi045 ceph-mon[33299]: from='client.? 172.21.15.32:0/3927348482' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-03T18:46:26.007 INFO:tasks.cephadm.ceph_manager.ceph:need seq 300647710748 got 300647710753 for osd.11 2024-05-03T18:46:26.007 DEBUG:teuthology.parallel:result is None 2024-05-03T18:46:26.240 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:46:26 smithi032 ceph-mon[29763]: from='client.? 172.21.15.32:0/456819420' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-03T18:46:26.497 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:46:26 smithi045 ceph-mon[33299]: from='client.? 172.21.15.32:0/456819420' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-03T18:46:26.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:46:26 smithi113 ceph-mon[33540]: from='client.? 172.21.15.32:0/456819420' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-03T18:46:27.399 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:46:27 smithi045 ceph-mon[33299]: pgmap v330: 97 pgs: 97 active+clean; 631 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:46:27.515 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:46:27 smithi032 ceph-mon[29763]: pgmap v330: 97 pgs: 97 active+clean; 631 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:46:27.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:46:27 smithi113 ceph-mon[33540]: pgmap v330: 97 pgs: 97 active+clean; 631 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:46:28.479 INFO:tasks.cephadm.ceph_manager.ceph:need seq 133143986240 got 133143986245 for osd.4 2024-05-03T18:46:28.479 DEBUG:teuthology.parallel:result is None 2024-05-03T18:46:28.659 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:28.658+0000 7f5f25066700 1 -- 172.21.15.32:0/2336827922 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5f20102fd0 msgr2=0x7f5f20107020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:28.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:28.658+0000 7f5f25066700 1 --2- 172.21.15.32:0/2336827922 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5f20102fd0 0x7f5f20107020 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f5f080099c0 tx=0x7f5f08009cd0 comp rx=0 tx=0).stop 2024-05-03T18:46:28.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:28.658+0000 7f5f25066700 1 -- 172.21.15.32:0/2336827922 shutdown_connections 2024-05-03T18:46:28.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:28.658+0000 7f5f25066700 1 --2- 172.21.15.32:0/2336827922 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5f20107700 0x7f5f2010f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:28.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:28.658+0000 7f5f25066700 1 --2- 172.21.15.32:0/2336827922 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5f20102620 0x7f5f20102a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:28.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:28.658+0000 7f5f25066700 1 --2- 172.21.15.32:0/2336827922 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5f20102fd0 0x7f5f20107020 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:28.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:28.658+0000 7f5f25066700 1 -- 172.21.15.32:0/2336827922 >> 172.21.15.32:0/2336827922 conn(0x7f5f20079440 msgr2=0x7f5f20079850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:46:28.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:28.658+0000 7f5f25066700 1 -- 172.21.15.32:0/2336827922 shutdown_connections 2024-05-03T18:46:28.660 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:28.658+0000 7f5f25066700 1 -- 172.21.15.32:0/2336827922 wait complete. 2024-05-03T18:46:28.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:28.659+0000 7f5f25066700 1 Processor -- start 2024-05-03T18:46:28.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:28.659+0000 7f5f25066700 1 -- start start 2024-05-03T18:46:28.662 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:28.659+0000 7f5f25066700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5f20102620 0x7f5f201040c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:28.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:28.659+0000 7f5f25066700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5f20102fd0 0x7f5f20104600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:28.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:28.659+0000 7f5f25066700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5f20107700 0x7f5f20104b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:28.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:28.659+0000 7f5f25066700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f5f201126c0 con 0x7f5f20102620 2024-05-03T18:46:28.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:28.659+0000 7f5f25066700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f5f20112540 con 0x7f5f20102fd0 2024-05-03T18:46:28.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:28.659+0000 7f5f25066700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f5f20112840 con 0x7f5f20107700 2024-05-03T18:46:28.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:28.660+0000 7f5f1ed9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5f20102620 0x7f5f201040c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:28.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:28.660+0000 7f5f1ed9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5f20102620 0x7f5f201040c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51516/0 (socket says 172.21.15.32:51516) 2024-05-03T18:46:28.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:28.660+0000 7f5f1ed9d700 1 -- 172.21.15.32:0/2971103852 learned_addr learned my addr 172.21.15.32:0/2971103852 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:46:28.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:28.660+0000 7f5f1f59e700 1 --2- 172.21.15.32:0/2971103852 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5f20107700 0x7f5f20104b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:28.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:28.660+0000 7f5f1e59c700 1 --2- 172.21.15.32:0/2971103852 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5f20102fd0 0x7f5f20104600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:28.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:28.660+0000 7f5f1ed9d700 1 -- 172.21.15.32:0/2971103852 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5f20107700 msgr2=0x7f5f20104b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:28.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:28.660+0000 7f5f1ed9d700 1 --2- 172.21.15.32:0/2971103852 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5f20107700 0x7f5f20104b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:28.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:28.660+0000 7f5f1ed9d700 1 -- 172.21.15.32:0/2971103852 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5f20102fd0 msgr2=0x7f5f20104600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:28.663 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:28.660+0000 7f5f1ed9d700 1 --2- 172.21.15.32:0/2971103852 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5f20102fd0 0x7f5f20104600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:28.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:28.660+0000 7f5f1ed9d700 1 -- 172.21.15.32:0/2971103852 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f201a8220 con 0x7f5f20102620 2024-05-03T18:46:28.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:28.660+0000 7f5f1e59c700 1 --2- 172.21.15.32:0/2971103852 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5f20102fd0 0x7f5f20104600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T18:46:28.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:28.660+0000 7f5f1ed9d700 1 --2- 172.21.15.32:0/2971103852 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5f20102620 0x7f5f201040c0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f5f1000b530 tx=0x7f5f1000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:46:28.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:28.660+0000 7f5f17fff700 1 -- 172.21.15.32:0/2971103852 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f10015020 con 0x7f5f20102620 2024-05-03T18:46:28.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:28.660+0000 7f5f17fff700 1 -- 172.21.15.32:0/2971103852 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f10011ce0 con 0x7f5f20102620 2024-05-03T18:46:28.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:28.660+0000 7f5f17fff700 1 -- 172.21.15.32:0/2971103852 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f10010e40 con 0x7f5f20102620 2024-05-03T18:46:28.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:28.660+0000 7f5f25066700 1 -- 172.21.15.32:0/2971103852 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f201a8510 con 0x7f5f20102620 2024-05-03T18:46:28.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:28.660+0000 7f5f25066700 1 -- 172.21.15.32:0/2971103852 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f201a89d0 con 0x7f5f20102620 2024-05-03T18:46:28.664 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:28.661+0000 7f5f25066700 1 -- 172.21.15.32:0/2971103852 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f2006a190 con 0x7f5f20102620 2024-05-03T18:46:28.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:28.663+0000 7f5f17fff700 1 -- 172.21.15.32:0/2971103852 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f5f10018020 con 0x7f5f20102620 2024-05-03T18:46:28.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:28.663+0000 7f5f17fff700 1 --2- 172.21.15.32:0/2971103852 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f5f0c0b2910 0x7f5f0c0b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:28.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:28.664+0000 7f5f1e59c700 1 --2- 172.21.15.32:0/2971103852 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f5f0c0b2910 0x7f5f0c0b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:28.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:28.664+0000 7f5f17fff700 1 -- 172.21.15.32:0/2971103852 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8269+0+0 (secure 0 0 0) 0x7f5f100e1ae0 con 0x7f5f20102620 2024-05-03T18:46:28.665 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:28.664+0000 7f5f1e59c700 1 --2- 172.21.15.32:0/2971103852 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f5f0c0b2910 0x7f5f0c0b4dd0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f5f08000c00 tx=0x7f5f08009630 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:46:28.669 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:28.668+0000 7f5f17fff700 1 -- 172.21.15.32:0/2971103852 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f5f1001f020 con 0x7f5f20102620 2024-05-03T18:46:28.883 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:28.882+0000 7f5f25066700 1 -- 172.21.15.32:0/2971103852 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f5f2002e510 con 0x7f5f20102620 2024-05-03T18:46:28.884 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:28.883+0000 7f5f17fff700 1 -- 172.21.15.32:0/2971103852 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f5f100a9790 con 0x7f5f20102620 2024-05-03T18:46:28.884 INFO:teuthology.orchestra.run.smithi032.stdout:227633266737 2024-05-03T18:46:28.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:28.885+0000 7f5f25066700 1 -- 172.21.15.32:0/2971103852 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f5f0c0b2910 msgr2=0x7f5f0c0b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:28.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:28.885+0000 7f5f25066700 1 --2- 172.21.15.32:0/2971103852 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f5f0c0b2910 0x7f5f0c0b4dd0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f5f08000c00 tx=0x7f5f08009630 comp rx=0 tx=0).stop 2024-05-03T18:46:28.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:28.885+0000 7f5f25066700 1 -- 172.21.15.32:0/2971103852 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5f20102620 msgr2=0x7f5f201040c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:28.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:28.885+0000 7f5f25066700 1 --2- 172.21.15.32:0/2971103852 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5f20102620 0x7f5f201040c0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f5f1000b530 tx=0x7f5f1000b840 comp rx=0 tx=0).stop 2024-05-03T18:46:28.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:28.885+0000 7f5f25066700 1 -- 172.21.15.32:0/2971103852 shutdown_connections 2024-05-03T18:46:28.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:28.885+0000 7f5f25066700 1 --2- 172.21.15.32:0/2971103852 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f5f0c0b2910 0x7f5f0c0b4dd0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:28.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:28.885+0000 7f5f25066700 1 --2- 172.21.15.32:0/2971103852 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5f20107700 0x7f5f20104b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:28.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:28.885+0000 7f5f25066700 1 --2- 172.21.15.32:0/2971103852 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5f20102620 0x7f5f201040c0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:28.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:28.885+0000 7f5f25066700 1 --2- 172.21.15.32:0/2971103852 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5f20102fd0 0x7f5f20104600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:28.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:28.885+0000 7f5f25066700 1 -- 172.21.15.32:0/2971103852 >> 172.21.15.32:0/2971103852 conn(0x7f5f20079440 msgr2=0x7f5f2006bf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:46:28.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:28.886+0000 7f5f25066700 1 -- 172.21.15.32:0/2971103852 shutdown_connections 2024-05-03T18:46:28.887 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:28.886+0000 7f5f25066700 1 -- 172.21.15.32:0/2971103852 wait complete. 2024-05-03T18:46:29.261 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:46:29 smithi032 ceph-mon[29763]: pgmap v331: 97 pgs: 97 active+clean; 631 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:46:29.262 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:46:29 smithi032 ceph-mon[29763]: from='client.? 172.21.15.32:0/2971103852' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-03T18:46:29.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:46:29 smithi045 ceph-mon[33299]: pgmap v331: 97 pgs: 97 active+clean; 631 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:46:29.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:46:29 smithi045 ceph-mon[33299]: from='client.? 172.21.15.32:0/2971103852' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-03T18:46:29.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:46:29 smithi113 ceph-mon[33540]: pgmap v331: 97 pgs: 97 active+clean; 631 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:46:29.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:46:29 smithi113 ceph-mon[33540]: from='client.? 172.21.15.32:0/2971103852' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-03T18:46:29.698 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:46:29.699 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:46:30.136 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:30.133+0000 7fbf08b9d700 1 -- 172.21.15.32:0/941998056 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf04102530 msgr2=0x7fbf04102910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:30.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:30.133+0000 7fbf08b9d700 1 --2- 172.21.15.32:0/941998056 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf04102530 0x7fbf04102910 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fbef40099c0 tx=0x7fbef4009cd0 comp rx=0 tx=0).stop 2024-05-03T18:46:30.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:30.134+0000 7fbf08b9d700 1 -- 172.21.15.32:0/941998056 shutdown_connections 2024-05-03T18:46:30.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:30.134+0000 7fbf08b9d700 1 --2- 172.21.15.32:0/941998056 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf04107610 0x7fbf0410f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:30.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:30.134+0000 7fbf08b9d700 1 --2- 172.21.15.32:0/941998056 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf04102530 0x7fbf04102910 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:30.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:30.134+0000 7fbf08b9d700 1 --2- 172.21.15.32:0/941998056 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbf04102ee0 0x7fbf04106f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:30.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:30.134+0000 7fbf08b9d700 1 -- 172.21.15.32:0/941998056 >> 172.21.15.32:0/941998056 conn(0x7fbf040fddc0 msgr2=0x7fbf041001e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:46:30.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:30.134+0000 7fbf08b9d700 1 -- 172.21.15.32:0/941998056 shutdown_connections 2024-05-03T18:46:30.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:30.134+0000 7fbf08b9d700 1 -- 172.21.15.32:0/941998056 wait complete. 2024-05-03T18:46:30.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:30.134+0000 7fbf08b9d700 1 Processor -- start 2024-05-03T18:46:30.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:30.135+0000 7fbf08b9d700 1 -- start start 2024-05-03T18:46:30.137 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:30.135+0000 7fbf08b9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf04102530 0x7fbf0419f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:30.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:30.135+0000 7fbf08b9d700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbf04102ee0 0x7fbf0419fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:30.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:30.135+0000 7fbf08b9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf04107610 0x7fbf041998b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:30.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:30.135+0000 7fbf08b9d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fbf041124c0 con 0x7fbf04102530 2024-05-03T18:46:30.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:30.135+0000 7fbf08b9d700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fbf04112340 con 0x7fbf04107610 2024-05-03T18:46:30.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:30.135+0000 7fbf08b9d700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fbf04112640 con 0x7fbf04102ee0 2024-05-03T18:46:30.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:30.135+0000 7fbf037fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf04102530 0x7fbf0419f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:30.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:30.135+0000 7fbf037fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf04102530 0x7fbf0419f6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51530/0 (socket says 172.21.15.32:51530) 2024-05-03T18:46:30.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:30.135+0000 7fbf037fe700 1 -- 172.21.15.32:0/3839927899 learned_addr learned my addr 172.21.15.32:0/3839927899 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:46:30.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:30.135+0000 7fbf02ffd700 1 --2- 172.21.15.32:0/3839927899 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbf04102ee0 0x7fbf0419fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:30.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:30.135+0000 7fbf037fe700 1 -- 172.21.15.32:0/3839927899 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbf04102ee0 msgr2=0x7fbf0419fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:30.138 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:30.135+0000 7fbf037fe700 1 --2- 172.21.15.32:0/3839927899 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbf04102ee0 0x7fbf0419fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:30.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:30.135+0000 7fbf03fff700 1 --2- 172.21.15.32:0/3839927899 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf04107610 0x7fbf041998b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:30.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:30.135+0000 7fbf037fe700 1 -- 172.21.15.32:0/3839927899 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf04107610 msgr2=0x7fbf041998b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:30.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:30.135+0000 7fbf037fe700 1 --2- 172.21.15.32:0/3839927899 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf04107610 0x7fbf041998b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:30.139 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:30.135+0000 7fbf037fe700 1 -- 172.21.15.32:0/3839927899 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf0419a010 con 0x7fbf04102530 2024-05-03T18:46:30.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:30.136+0000 7fbf037fe700 1 --2- 172.21.15.32:0/3839927899 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf04102530 0x7fbf0419f6f0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fbef400b580 tx=0x7fbef40047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:46:30.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:30.136+0000 7fbf00ff9700 1 -- 172.21.15.32:0/3839927899 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbef4027070 con 0x7fbf04102530 2024-05-03T18:46:30.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:30.136+0000 7fbf00ff9700 1 -- 172.21.15.32:0/3839927899 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbef4004020 con 0x7fbf04102530 2024-05-03T18:46:30.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:30.136+0000 7fbf00ff9700 1 -- 172.21.15.32:0/3839927899 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbef4022e30 con 0x7fbf04102530 2024-05-03T18:46:30.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:30.136+0000 7fbf08b9d700 1 -- 172.21.15.32:0/3839927899 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf0419a2a0 con 0x7fbf04102530 2024-05-03T18:46:30.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:30.136+0000 7fbf08b9d700 1 -- 172.21.15.32:0/3839927899 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf041a65c0 con 0x7fbf04102530 2024-05-03T18:46:30.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:30.137+0000 7fbf08b9d700 1 -- 172.21.15.32:0/3839927899 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf04103a10 con 0x7fbf04102530 2024-05-03T18:46:30.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:30.139+0000 7fbf00ff9700 1 -- 172.21.15.32:0/3839927899 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fbef4022590 con 0x7fbf04102530 2024-05-03T18:46:30.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:30.139+0000 7fbf00ff9700 1 --2- 172.21.15.32:0/3839927899 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fbeec0b2990 0x7fbeec0b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:30.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:30.139+0000 7fbf02ffd700 1 --2- 172.21.15.32:0/3839927899 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fbeec0b2990 0x7fbeec0b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:30.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:30.140+0000 7fbf00ff9700 1 -- 172.21.15.32:0/3839927899 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8269+0+0 (secure 0 0 0) 0x7fbef40e3440 con 0x7fbf04102530 2024-05-03T18:46:30.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:30.140+0000 7fbf02ffd700 1 --2- 172.21.15.32:0/3839927899 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fbeec0b2990 0x7fbeec0b4e50 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fbef8005950 tx=0x7fbef80058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:46:30.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:30.143+0000 7fbf00ff9700 1 -- 172.21.15.32:0/3839927899 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fbef4030070 con 0x7fbf04102530 2024-05-03T18:46:30.363 INFO:teuthology.orchestra.run.smithi032.stdout:154618822720 2024-05-03T18:46:30.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:30.361+0000 7fbf08b9d700 1 -- 172.21.15.32:0/3839927899 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fbf0402e510 con 0x7fbf04102530 2024-05-03T18:46:30.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:30.362+0000 7fbf00ff9700 1 -- 172.21.15.32:0/3839927899 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fbef4013070 con 0x7fbf04102530 2024-05-03T18:46:30.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:30.365+0000 7fbf08b9d700 1 -- 172.21.15.32:0/3839927899 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fbeec0b2990 msgr2=0x7fbeec0b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:30.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:30.365+0000 7fbf08b9d700 1 --2- 172.21.15.32:0/3839927899 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fbeec0b2990 0x7fbeec0b4e50 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fbef8005950 tx=0x7fbef80058e0 comp rx=0 tx=0).stop 2024-05-03T18:46:30.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:30.365+0000 7fbf08b9d700 1 -- 172.21.15.32:0/3839927899 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf04102530 msgr2=0x7fbf0419f6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:30.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:30.365+0000 7fbf08b9d700 1 --2- 172.21.15.32:0/3839927899 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf04102530 0x7fbf0419f6f0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fbef400b580 tx=0x7fbef40047b0 comp rx=0 tx=0).stop 2024-05-03T18:46:30.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:30.366+0000 7fbf08b9d700 1 -- 172.21.15.32:0/3839927899 shutdown_connections 2024-05-03T18:46:30.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:30.366+0000 7fbf08b9d700 1 --2- 172.21.15.32:0/3839927899 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fbeec0b2990 0x7fbeec0b4e50 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:30.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:30.366+0000 7fbf08b9d700 1 --2- 172.21.15.32:0/3839927899 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbf04107610 0x7fbf041998b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:30.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:30.366+0000 7fbf08b9d700 1 --2- 172.21.15.32:0/3839927899 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf04102530 0x7fbf0419f6f0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:30.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:30.366+0000 7fbf08b9d700 1 --2- 172.21.15.32:0/3839927899 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbf04102ee0 0x7fbf0419fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:30.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:30.366+0000 7fbf08b9d700 1 -- 172.21.15.32:0/3839927899 >> 172.21.15.32:0/3839927899 conn(0x7fbf040fddc0 msgr2=0x7fbf041064b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:46:30.367 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:30.366+0000 7fbf08b9d700 1 -- 172.21.15.32:0/3839927899 shutdown_connections 2024-05-03T18:46:30.368 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:30.366+0000 7fbf08b9d700 1 -- 172.21.15.32:0/3839927899 wait complete. 2024-05-03T18:46:30.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:30.965+0000 7f27c89ae700 1 -- 172.21.15.32:0/2932230941 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f27c4101a90 msgr2=0x7f27c4101f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:30.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:30.965+0000 7f27c89ae700 1 --2- 172.21.15.32:0/2932230941 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f27c4101a90 0x7f27c4101f10 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f27b4009990 tx=0x7f27b4009ca0 comp rx=0 tx=0).stop 2024-05-03T18:46:30.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:30.965+0000 7f27c89ae700 1 -- 172.21.15.32:0/2932230941 shutdown_connections 2024-05-03T18:46:30.968 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:30.965+0000 7f27c89ae700 1 --2- 172.21.15.32:0/2932230941 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f27c4102450 0x7f27c4110340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:30.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:30.965+0000 7f27c89ae700 1 --2- 172.21.15.32:0/2932230941 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27c4109520 0x7f27c4109900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:30.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:30.965+0000 7f27c89ae700 1 --2- 172.21.15.32:0/2932230941 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f27c4101a90 0x7f27c4101f10 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:30.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:30.965+0000 7f27c89ae700 1 -- 172.21.15.32:0/2932230941 >> 172.21.15.32:0/2932230941 conn(0x7f27c4078a10 msgr2=0x7f27c40ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:46:30.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:30.965+0000 7f27c89ae700 1 -- 172.21.15.32:0/2932230941 shutdown_connections 2024-05-03T18:46:30.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:30.965+0000 7f27c89ae700 1 -- 172.21.15.32:0/2932230941 wait complete. 2024-05-03T18:46:30.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:30.966+0000 7f27c89ae700 1 Processor -- start 2024-05-03T18:46:30.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:30.966+0000 7f27c89ae700 1 -- start start 2024-05-03T18:46:30.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:30.966+0000 7f27c89ae700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f27c4101a90 0x7f27c419f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:30.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:30.966+0000 7f27c89ae700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27c4102450 0x7f27c419fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:30.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:30.966+0000 7f27c89ae700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f27c4109520 0x7f27c4199990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:30.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:30.966+0000 7f27c89ae700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f27c4113650 con 0x7f27c4101a90 2024-05-03T18:46:30.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:30.966+0000 7f27c89ae700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f27c41134d0 con 0x7f27c4102450 2024-05-03T18:46:30.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:30.966+0000 7f27c89ae700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f27c41137d0 con 0x7f27c4109520 2024-05-03T18:46:30.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:30.966+0000 7f27c1d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27c4102450 0x7f27c419fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:30.969 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:30.966+0000 7f27c1d9b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27c4102450 0x7f27c419fd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.32:52134/0 (socket says 172.21.15.32:52134) 2024-05-03T18:46:30.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:30.966+0000 7f27c1d9b700 1 -- 172.21.15.32:0/2890676110 learned_addr learned my addr 172.21.15.32:0/2890676110 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:46:30.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:30.966+0000 7f27c259c700 1 --2- 172.21.15.32:0/2890676110 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f27c4101a90 0x7f27c419f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:30.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:30.966+0000 7f27c2d9d700 1 --2- 172.21.15.32:0/2890676110 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f27c4109520 0x7f27c4199990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:30.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:30.967+0000 7f27c1d9b700 1 -- 172.21.15.32:0/2890676110 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f27c4109520 msgr2=0x7f27c4199990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:30.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:30.967+0000 7f27c1d9b700 1 --2- 172.21.15.32:0/2890676110 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f27c4109520 0x7f27c4199990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:30.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:30.967+0000 7f27c1d9b700 1 -- 172.21.15.32:0/2890676110 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f27c4101a90 msgr2=0x7f27c419f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:30.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:30.967+0000 7f27c1d9b700 1 --2- 172.21.15.32:0/2890676110 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f27c4101a90 0x7f27c419f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:30.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:30.967+0000 7f27c1d9b700 1 -- 172.21.15.32:0/2890676110 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27c419a0f0 con 0x7f27c4102450 2024-05-03T18:46:30.970 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:30.967+0000 7f27c259c700 1 --2- 172.21.15.32:0/2890676110 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f27c4101a90 0x7f27c419f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T18:46:30.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:30.967+0000 7f27c1d9b700 1 --2- 172.21.15.32:0/2890676110 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27c4102450 0x7f27c419fd10 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f27b4000c00 tx=0x7f27b40047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:46:30.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:30.967+0000 7f27b37fe700 1 -- 172.21.15.32:0/2890676110 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27b4027070 con 0x7f27c4102450 2024-05-03T18:46:30.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:30.967+0000 7f27c89ae700 1 -- 172.21.15.32:0/2890676110 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27c419a380 con 0x7f27c4102450 2024-05-03T18:46:30.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:30.967+0000 7f27b37fe700 1 -- 172.21.15.32:0/2890676110 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27b4003f20 con 0x7f27c4102450 2024-05-03T18:46:30.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:30.967+0000 7f27b37fe700 1 -- 172.21.15.32:0/2890676110 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27b4022ca0 con 0x7f27c4102450 2024-05-03T18:46:30.971 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:30.967+0000 7f27c89ae700 1 -- 172.21.15.32:0/2890676110 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27c41a66a0 con 0x7f27c4102450 2024-05-03T18:46:30.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:30.968+0000 7f27c89ae700 1 -- 172.21.15.32:0/2890676110 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27c406a190 con 0x7f27c4102450 2024-05-03T18:46:30.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:30.970+0000 7f27b37fe700 1 -- 172.21.15.32:0/2890676110 <== mon.2 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f27b402a070 con 0x7f27c4102450 2024-05-03T18:46:30.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:30.970+0000 7f27b37fe700 1 --2- 172.21.15.32:0/2890676110 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f27ac0b2910 0x7f27ac0b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:30.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:30.970+0000 7f27c259c700 1 --2- 172.21.15.32:0/2890676110 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f27ac0b2910 0x7f27ac0b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:30.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:30.971+0000 7f27b37fe700 1 -- 172.21.15.32:0/2890676110 <== mon.2 v2:172.21.15.45:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8269+0+0 (secure 0 0 0) 0x7f27b40e3680 con 0x7f27c4102450 2024-05-03T18:46:30.972 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:30.971+0000 7f27c259c700 1 --2- 172.21.15.32:0/2890676110 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f27ac0b2910 0x7f27ac0b4dd0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f27c419aad0 tx=0x7f27b800a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:46:30.975 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:30.974+0000 7f27b37fe700 1 -- 172.21.15.32:0/2890676110 <== mon.2 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f27b4031070 con 0x7f27c4102450 2024-05-03T18:46:31.174 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:46:31 smithi032 ceph-mon[29763]: pgmap v332: 97 pgs: 97 active+clean; 631 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:46:31.175 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:46:31 smithi032 ceph-mon[29763]: from='client.? 172.21.15.32:0/3839927899' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-03T18:46:31.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:31.208+0000 7f27c89ae700 1 -- 172.21.15.32:0/2890676110 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f27c402e510 con 0x7f27c4102450 2024-05-03T18:46:31.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:31.208+0000 7f27b37fe700 1 -- 172.21.15.32:0/2890676110 <== mon.2 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f27b4013070 con 0x7f27c4102450 2024-05-03T18:46:31.209 INFO:teuthology.orchestra.run.smithi032.stdout:77309411408 2024-05-03T18:46:31.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:31.210+0000 7f27c89ae700 1 -- 172.21.15.32:0/2890676110 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f27ac0b2910 msgr2=0x7f27ac0b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:31.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:31.210+0000 7f27c89ae700 1 --2- 172.21.15.32:0/2890676110 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f27ac0b2910 0x7f27ac0b4dd0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f27c419aad0 tx=0x7f27b800a380 comp rx=0 tx=0).stop 2024-05-03T18:46:31.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:31.210+0000 7f27c89ae700 1 -- 172.21.15.32:0/2890676110 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27c4102450 msgr2=0x7f27c419fd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:31.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:31.210+0000 7f27c89ae700 1 --2- 172.21.15.32:0/2890676110 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27c4102450 0x7f27c419fd10 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f27b4000c00 tx=0x7f27b40047b0 comp rx=0 tx=0).stop 2024-05-03T18:46:31.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:31.210+0000 7f27c89ae700 1 -- 172.21.15.32:0/2890676110 shutdown_connections 2024-05-03T18:46:31.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:31.210+0000 7f27c89ae700 1 --2- 172.21.15.32:0/2890676110 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f27ac0b2910 0x7f27ac0b4dd0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:31.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:31.210+0000 7f27c89ae700 1 --2- 172.21.15.32:0/2890676110 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f27c4109520 0x7f27c4199990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:31.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:31.210+0000 7f27c89ae700 1 --2- 172.21.15.32:0/2890676110 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f27c4101a90 0x7f27c419f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:31.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:31.210+0000 7f27c89ae700 1 --2- 172.21.15.32:0/2890676110 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27c4102450 0x7f27c419fd10 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:31.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:31.210+0000 7f27c89ae700 1 -- 172.21.15.32:0/2890676110 >> 172.21.15.32:0/2890676110 conn(0x7f27c4078a10 msgr2=0x7f27c4100150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:46:31.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:31.211+0000 7f27c89ae700 1 -- 172.21.15.32:0/2890676110 shutdown_connections 2024-05-03T18:46:31.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:31.211+0000 7f27c89ae700 1 -- 172.21.15.32:0/2890676110 wait complete. 2024-05-03T18:46:31.497 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:46:31 smithi045 ceph-mon[33299]: pgmap v332: 97 pgs: 97 active+clean; 631 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:46:31.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:46:31 smithi045 ceph-mon[33299]: from='client.? 172.21.15.32:0/3839927899' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-03T18:46:31.528 INFO:tasks.cephadm.ceph_manager.ceph:need seq 154618822717 got 154618822720 for osd.5 2024-05-03T18:46:31.528 DEBUG:teuthology.parallel:result is None 2024-05-03T18:46:31.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:46:31 smithi113 ceph-mon[33540]: pgmap v332: 97 pgs: 97 active+clean; 631 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:46:31.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:46:31 smithi113 ceph-mon[33540]: from='client.? 172.21.15.32:0/3839927899' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-03T18:46:32.031 INFO:tasks.cephadm.ceph_manager.ceph:need seq 227633266732 got 227633266737 for osd.8 2024-05-03T18:46:32.031 DEBUG:teuthology.parallel:result is None 2024-05-03T18:46:32.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:46:32 smithi032 ceph-mon[29763]: from='client.? 172.21.15.32:0/2890676110' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-03T18:46:32.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:46:32 smithi045 ceph-mon[33299]: from='client.? 172.21.15.32:0/2890676110' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-03T18:46:32.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:46:32 smithi113 ceph-mon[33540]: from='client.? 172.21.15.32:0/2890676110' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-03T18:46:33.418 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:46:33 smithi032 ceph-mon[29763]: pgmap v333: 97 pgs: 97 active+clean; 631 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:46:33.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:46:33 smithi045 ceph-mon[33299]: pgmap v333: 97 pgs: 97 active+clean; 631 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:46:33.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:46:33 smithi113 ceph-mon[33540]: pgmap v333: 97 pgs: 97 active+clean; 631 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:46:34.553 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411403 got 77309411408 for osd.2 2024-05-03T18:46:34.553 DEBUG:teuthology.parallel:result is None 2024-05-03T18:46:35.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:35.059+0000 7f19027bc700 1 -- 172.21.15.32:0/853180919 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18fc102450 msgr2=0x7f18fc110340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:35.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:35.059+0000 7f19027bc700 1 --2- 172.21.15.32:0/853180919 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18fc102450 0x7f18fc110340 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f18f400b0a0 tx=0x7f18f400b3b0 comp rx=0 tx=0).stop 2024-05-03T18:46:35.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:35.059+0000 7f19027bc700 1 -- 172.21.15.32:0/853180919 shutdown_connections 2024-05-03T18:46:35.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:35.059+0000 7f19027bc700 1 --2- 172.21.15.32:0/853180919 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18fc102450 0x7f18fc110340 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:35.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:35.059+0000 7f19027bc700 1 --2- 172.21.15.32:0/853180919 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18fc109520 0x7f18fc109900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:35.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:35.059+0000 7f19027bc700 1 --2- 172.21.15.32:0/853180919 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18fc101a90 0x7f18fc101f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:35.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:35.059+0000 7f19027bc700 1 -- 172.21.15.32:0/853180919 >> 172.21.15.32:0/853180919 conn(0x7f18fc078a10 msgr2=0x7f18fc0ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:46:35.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:35.059+0000 7f19027bc700 1 -- 172.21.15.32:0/853180919 shutdown_connections 2024-05-03T18:46:35.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:35.059+0000 7f19027bc700 1 -- 172.21.15.32:0/853180919 wait complete. 2024-05-03T18:46:35.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:35.060+0000 7f19027bc700 1 Processor -- start 2024-05-03T18:46:35.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:35.060+0000 7f19027bc700 1 -- start start 2024-05-03T18:46:35.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:35.060+0000 7f19027bc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18fc101a90 0x7f18fc19f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:35.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:35.060+0000 7f19027bc700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18fc102450 0x7f18fc19fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:35.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:35.060+0000 7f19027bc700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18fc109520 0x7f18fc1998f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:35.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:35.060+0000 7f19027bc700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f18fc113630 con 0x7f18fc101a90 2024-05-03T18:46:35.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:35.060+0000 7f19027bc700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f18fc1134b0 con 0x7f18fc109520 2024-05-03T18:46:35.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:35.060+0000 7f19027bc700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f18fc1137b0 con 0x7f18fc102450 2024-05-03T18:46:35.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:35.061+0000 7f18fbfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18fc101a90 0x7f18fc19f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:35.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:35.061+0000 7f18fbfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18fc101a90 0x7f18fc19f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46498/0 (socket says 172.21.15.32:46498) 2024-05-03T18:46:35.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:35.061+0000 7f18fbfff700 1 -- 172.21.15.32:0/3560565726 learned_addr learned my addr 172.21.15.32:0/3560565726 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:46:35.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:35.061+0000 7f18fb7fe700 1 --2- 172.21.15.32:0/3560565726 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18fc102450 0x7f18fc19fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:35.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:35.061+0000 7f1900d59700 1 --2- 172.21.15.32:0/3560565726 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18fc109520 0x7f18fc1998f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:35.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:35.061+0000 7f18fbfff700 1 -- 172.21.15.32:0/3560565726 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18fc102450 msgr2=0x7f18fc19fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:35.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:35.061+0000 7f18fbfff700 1 --2- 172.21.15.32:0/3560565726 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18fc102450 0x7f18fc19fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:35.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:35.061+0000 7f18fbfff700 1 -- 172.21.15.32:0/3560565726 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18fc109520 msgr2=0x7f18fc1998f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:35.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:35.061+0000 7f18fbfff700 1 --2- 172.21.15.32:0/3560565726 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18fc109520 0x7f18fc1998f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:35.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:35.061+0000 7f18fbfff700 1 -- 172.21.15.32:0/3560565726 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18fc19a050 con 0x7f18fc101a90 2024-05-03T18:46:35.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:35.061+0000 7f18fbfff700 1 --2- 172.21.15.32:0/3560565726 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18fc101a90 0x7f18fc19f730 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f18ec00d950 tx=0x7f18ec00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:46:35.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:35.061+0000 7f18f97fa700 1 -- 172.21.15.32:0/3560565726 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18ec016070 con 0x7f18fc101a90 2024-05-03T18:46:35.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:35.061+0000 7f18f97fa700 1 -- 172.21.15.32:0/3560565726 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18ec00fd40 con 0x7f18fc101a90 2024-05-03T18:46:35.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:35.061+0000 7f19027bc700 1 -- 172.21.15.32:0/3560565726 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18fc19a340 con 0x7f18fc101a90 2024-05-03T18:46:35.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:35.062+0000 7f19027bc700 1 -- 172.21.15.32:0/3560565726 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18fc1a6600 con 0x7f18fc101a90 2024-05-03T18:46:35.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:35.062+0000 7f18f97fa700 1 -- 172.21.15.32:0/3560565726 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18ec00e3b0 con 0x7f18fc101a90 2024-05-03T18:46:35.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:35.063+0000 7f19027bc700 1 -- 172.21.15.32:0/3560565726 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18fc06a190 con 0x7f18fc101a90 2024-05-03T18:46:35.066 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:35.065+0000 7f18f97fa700 1 -- 172.21.15.32:0/3560565726 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f18ec019070 con 0x7f18fc101a90 2024-05-03T18:46:35.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:35.066+0000 7f18f97fa700 1 --2- 172.21.15.32:0/3560565726 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f18e40b2990 0x7f18e40b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:35.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:35.066+0000 7f18fb7fe700 1 --2- 172.21.15.32:0/3560565726 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f18e40b2990 0x7f18e40b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:35.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:35.066+0000 7f18f97fa700 1 -- 172.21.15.32:0/3560565726 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8269+0+0 (secure 0 0 0) 0x7f18ec0e2160 con 0x7f18fc101a90 2024-05-03T18:46:35.067 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:35.066+0000 7f18fb7fe700 1 --2- 172.21.15.32:0/3560565726 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f18e40b2990 0x7f18e40b4e50 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f18fc108c70 tx=0x7f18f000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:46:35.070 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:35.069+0000 7f18f97fa700 1 -- 172.21.15.32:0/3560565726 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f18ec0a9ca0 con 0x7f18fc101a90 2024-05-03T18:46:35.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:34.947+0000 7f5c488a3700 1 -- 172.21.15.32:0/350810309 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c441023a0 msgr2=0x7f5c44102820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:35.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:34.947+0000 7f5c488a3700 1 --2- 172.21.15.32:0/350810309 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c441023a0 0x7f5c44102820 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f5c2c009990 tx=0x7f5c2c009ca0 comp rx=0 tx=0).stop 2024-05-03T18:46:35.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:34.947+0000 7f5c488a3700 1 -- 172.21.15.32:0/350810309 shutdown_connections 2024-05-03T18:46:35.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:34.947+0000 7f5c488a3700 1 --2- 172.21.15.32:0/350810309 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5c44102d60 0x7f5c4410f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:35.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:34.947+0000 7f5c488a3700 1 --2- 172.21.15.32:0/350810309 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c44108410 0x7f5c441087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:35.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:34.947+0000 7f5c488a3700 1 --2- 172.21.15.32:0/350810309 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c441023a0 0x7f5c44102820 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:35.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:34.947+0000 7f5c488a3700 1 -- 172.21.15.32:0/350810309 >> 172.21.15.32:0/350810309 conn(0x7f5c44079350 msgr2=0x7f5c44079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:46:35.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:34.947+0000 7f5c488a3700 1 -- 172.21.15.32:0/350810309 shutdown_connections 2024-05-03T18:46:35.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:34.947+0000 7f5c488a3700 1 -- 172.21.15.32:0/350810309 wait complete. 2024-05-03T18:46:35.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:34.948+0000 7f5c488a3700 1 Processor -- start 2024-05-03T18:46:35.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:34.948+0000 7f5c488a3700 1 -- start start 2024-05-03T18:46:35.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:34.948+0000 7f5c488a3700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c441023a0 0x7f5c4419bed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:35.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:34.948+0000 7f5c488a3700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5c44102d60 0x7f5c4419c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:35.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:34.948+0000 7f5c488a3700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c44108410 0x7f5c441a07a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:35.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:34.948+0000 7f5c488a3700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f5c44112630 con 0x7f5c441023a0 2024-05-03T18:46:35.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:34.948+0000 7f5c488a3700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f5c441124b0 con 0x7f5c44108410 2024-05-03T18:46:35.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:34.948+0000 7f5c488a3700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f5c441127b0 con 0x7f5c44102d60 2024-05-03T18:46:35.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:34.948+0000 7f5c4259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c441023a0 0x7f5c4419bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:35.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:34.948+0000 7f5c4259c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c441023a0 0x7f5c4419bed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46482/0 (socket says 172.21.15.32:46482) 2024-05-03T18:46:35.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:34.948+0000 7f5c4259c700 1 -- 172.21.15.32:0/3877665801 learned_addr learned my addr 172.21.15.32:0/3877665801 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:46:35.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:34.948+0000 7f5c41d9b700 1 --2- 172.21.15.32:0/3877665801 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5c44102d60 0x7f5c4419c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:35.209 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:34.948+0000 7f5c42d9d700 1 --2- 172.21.15.32:0/3877665801 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c44108410 0x7f5c441a07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:35.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:34.948+0000 7f5c4259c700 1 -- 172.21.15.32:0/3877665801 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5c44102d60 msgr2=0x7f5c4419c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:35.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:34.948+0000 7f5c4259c700 1 --2- 172.21.15.32:0/3877665801 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5c44102d60 0x7f5c4419c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:35.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:34.948+0000 7f5c4259c700 1 -- 172.21.15.32:0/3877665801 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c44108410 msgr2=0x7f5c441a07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:35.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:34.948+0000 7f5c4259c700 1 --2- 172.21.15.32:0/3877665801 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c44108410 0x7f5c441a07a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:35.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:34.948+0000 7f5c4259c700 1 -- 172.21.15.32:0/3877665801 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c441a0ee0 con 0x7f5c441023a0 2024-05-03T18:46:35.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:34.949+0000 7f5c4259c700 1 --2- 172.21.15.32:0/3877665801 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c441023a0 0x7f5c4419bed0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f5c3c00ba50 tx=0x7f5c3c00be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:46:35.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:34.949+0000 7f5c3b7fe700 1 -- 172.21.15.32:0/3877665801 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c3c019070 con 0x7f5c441023a0 2024-05-03T18:46:35.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:34.949+0000 7f5c3b7fe700 1 -- 172.21.15.32:0/3877665801 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5c3c014440 con 0x7f5c441023a0 2024-05-03T18:46:35.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:34.949+0000 7f5c488a3700 1 -- 172.21.15.32:0/3877665801 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c441a11d0 con 0x7f5c441023a0 2024-05-03T18:46:35.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:34.949+0000 7f5c3b7fe700 1 -- 172.21.15.32:0/3877665801 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c3c0075d0 con 0x7f5c441023a0 2024-05-03T18:46:35.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:34.949+0000 7f5c488a3700 1 -- 172.21.15.32:0/3877665801 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c441a8a50 con 0x7f5c441023a0 2024-05-03T18:46:35.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:34.950+0000 7f5c488a3700 1 -- 172.21.15.32:0/3877665801 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c4406a190 con 0x7f5c441023a0 2024-05-03T18:46:35.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:34.951+0000 7f5c3b7fe700 1 -- 172.21.15.32:0/3877665801 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f5c3c01c070 con 0x7f5c441023a0 2024-05-03T18:46:35.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:34.952+0000 7f5c3b7fe700 1 --2- 172.21.15.32:0/3877665801 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f5c300b2990 0x7f5c300b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:35.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:34.952+0000 7f5c41d9b700 1 --2- 172.21.15.32:0/3877665801 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f5c300b2990 0x7f5c300b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:35.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:34.952+0000 7f5c3b7fe700 1 -- 172.21.15.32:0/3877665801 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8269+0+0 (secure 0 0 0) 0x7f5c3c0e1b60 con 0x7f5c441023a0 2024-05-03T18:46:35.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:34.952+0000 7f5c41d9b700 1 --2- 172.21.15.32:0/3877665801 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f5c300b2990 0x7f5c300b4e50 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f5c2c006010 tx=0x7f5c2c01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:46:35.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:34.956+0000 7f5c3b7fe700 1 -- 172.21.15.32:0/3877665801 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f5c3c0a9810 con 0x7f5c441023a0 2024-05-03T18:46:35.211 INFO:teuthology.orchestra.run.smithi032.stdout:253403070509 2024-05-03T18:46:35.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:35.181+0000 7f5c488a3700 1 -- 172.21.15.32:0/3877665801 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f5c4419d230 con 0x7f5c441023a0 2024-05-03T18:46:35.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:35.182+0000 7f5c3b7fe700 1 -- 172.21.15.32:0/3877665801 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f5c4419d230 con 0x7f5c441023a0 2024-05-03T18:46:35.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:35.184+0000 7f5c488a3700 1 -- 172.21.15.32:0/3877665801 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f5c300b2990 msgr2=0x7f5c300b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:35.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:35.184+0000 7f5c488a3700 1 --2- 172.21.15.32:0/3877665801 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f5c300b2990 0x7f5c300b4e50 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f5c2c006010 tx=0x7f5c2c01a040 comp rx=0 tx=0).stop 2024-05-03T18:46:35.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:35.184+0000 7f5c488a3700 1 -- 172.21.15.32:0/3877665801 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c441023a0 msgr2=0x7f5c4419bed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:35.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:35.184+0000 7f5c488a3700 1 --2- 172.21.15.32:0/3877665801 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c441023a0 0x7f5c4419bed0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f5c3c00ba50 tx=0x7f5c3c00be10 comp rx=0 tx=0).stop 2024-05-03T18:46:35.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:35.184+0000 7f5c488a3700 1 -- 172.21.15.32:0/3877665801 shutdown_connections 2024-05-03T18:46:35.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:35.184+0000 7f5c488a3700 1 --2- 172.21.15.32:0/3877665801 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f5c300b2990 0x7f5c300b4e50 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:35.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:35.184+0000 7f5c488a3700 1 --2- 172.21.15.32:0/3877665801 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5c44108410 0x7f5c441a07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:35.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:35.184+0000 7f5c488a3700 1 --2- 172.21.15.32:0/3877665801 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c441023a0 0x7f5c4419bed0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:35.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:35.184+0000 7f5c488a3700 1 --2- 172.21.15.32:0/3877665801 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5c44102d60 0x7f5c4419c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:35.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:35.184+0000 7f5c488a3700 1 -- 172.21.15.32:0/3877665801 >> 172.21.15.32:0/3877665801 conn(0x7f5c44079350 msgr2=0x7f5c4406b1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:46:35.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:35.184+0000 7f5c488a3700 1 -- 172.21.15.32:0/3877665801 shutdown_connections 2024-05-03T18:46:35.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:35.184+0000 7f5c488a3700 1 -- 172.21.15.32:0/3877665801 wait complete. 2024-05-03T18:46:35.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:35.294+0000 7f19027bc700 1 -- 172.21.15.32:0/3560565726 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f18fc19ae20 con 0x7f18fc101a90 2024-05-03T18:46:35.295 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:35.294+0000 7f18f97fa700 1 -- 172.21.15.32:0/3560565726 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f18ec0a95a0 con 0x7f18fc101a90 2024-05-03T18:46:35.296 INFO:teuthology.orchestra.run.smithi032.stdout:34359738457 2024-05-03T18:46:35.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:35.297+0000 7f19027bc700 1 -- 172.21.15.32:0/3560565726 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f18e40b2990 msgr2=0x7f18e40b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:35.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:35.297+0000 7f19027bc700 1 --2- 172.21.15.32:0/3560565726 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f18e40b2990 0x7f18e40b4e50 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f18fc108c70 tx=0x7f18f000b410 comp rx=0 tx=0).stop 2024-05-03T18:46:35.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:35.297+0000 7f19027bc700 1 -- 172.21.15.32:0/3560565726 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18fc101a90 msgr2=0x7f18fc19f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:35.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:35.297+0000 7f19027bc700 1 --2- 172.21.15.32:0/3560565726 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18fc101a90 0x7f18fc19f730 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f18ec00d950 tx=0x7f18ec00dc60 comp rx=0 tx=0).stop 2024-05-03T18:46:35.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:35.297+0000 7f19027bc700 1 -- 172.21.15.32:0/3560565726 shutdown_connections 2024-05-03T18:46:35.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:35.297+0000 7f19027bc700 1 --2- 172.21.15.32:0/3560565726 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f18e40b2990 0x7f18e40b4e50 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:35.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:35.297+0000 7f19027bc700 1 --2- 172.21.15.32:0/3560565726 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18fc109520 0x7f18fc1998f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:35.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:35.297+0000 7f19027bc700 1 --2- 172.21.15.32:0/3560565726 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18fc101a90 0x7f18fc19f730 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:35.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:35.297+0000 7f19027bc700 1 --2- 172.21.15.32:0/3560565726 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18fc102450 0x7f18fc19fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:35.298 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:35.297+0000 7f19027bc700 1 -- 172.21.15.32:0/3560565726 >> 172.21.15.32:0/3560565726 conn(0x7f18fc078a10 msgr2=0x7f18fc100260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:46:35.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:35.297+0000 7f19027bc700 1 -- 172.21.15.32:0/3560565726 shutdown_connections 2024-05-03T18:46:35.299 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:35.297+0000 7f19027bc700 1 -- 172.21.15.32:0/3560565726 wait complete. 2024-05-03T18:46:35.413 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:46:35 smithi032 ceph-mon[29763]: pgmap v334: 97 pgs: 97 active+clean; 631 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:46:35.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:46:35 smithi045 ceph-mon[33299]: pgmap v334: 97 pgs: 97 active+clean; 631 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:46:35.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:46:35 smithi113 ceph-mon[33540]: pgmap v334: 97 pgs: 97 active+clean; 631 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:46:36.089 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738453 got 34359738457 for osd.0 2024-05-03T18:46:36.090 DEBUG:teuthology.parallel:result is None 2024-05-03T18:46:36.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:46:36 smithi032 ceph-mon[29763]: from='client.? 172.21.15.32:0/3877665801' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-03T18:46:36.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:46:36 smithi032 ceph-mon[29763]: from='client.? 172.21.15.32:0/3560565726' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-03T18:46:36.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:46:36 smithi045 ceph-mon[33299]: from='client.? 172.21.15.32:0/3877665801' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-03T18:46:36.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:46:36 smithi045 ceph-mon[33299]: from='client.? 172.21.15.32:0/3560565726' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-03T18:46:36.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:46:36 smithi113 ceph-mon[33540]: from='client.? 172.21.15.32:0/3877665801' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-03T18:46:36.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:46:36 smithi113 ceph-mon[33540]: from='client.? 172.21.15.32:0/3560565726' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-03T18:46:36.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:36.533+0000 7f99e8c88700 1 -- 172.21.15.32:0/2404081877 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99e410fed0 msgr2=0x7f99e41122c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:36.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:36.533+0000 7f99e8c88700 1 --2- 172.21.15.32:0/2404081877 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99e410fed0 0x7f99e41122c0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f99d4009990 tx=0x7f99d4009ca0 comp rx=0 tx=0).stop 2024-05-03T18:46:36.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:36.533+0000 7f99e8c88700 1 -- 172.21.15.32:0/2404081877 shutdown_connections 2024-05-03T18:46:36.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:36.533+0000 7f99e8c88700 1 --2- 172.21.15.32:0/2404081877 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99e410fed0 0x7f99e41122c0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:36.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:36.533+0000 7f99e8c88700 1 --2- 172.21.15.32:0/2404081877 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f99e40756d0 0x7f99e4075ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:36.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:36.533+0000 7f99e8c88700 1 --2- 172.21.15.32:0/2404081877 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f99e4075ff0 0x7f99e410f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:36.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:36.533+0000 7f99e8c88700 1 -- 172.21.15.32:0/2404081877 >> 172.21.15.32:0/2404081877 conn(0x7f99e407a650 msgr2=0x7f99e407aa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:46:36.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:36.533+0000 7f99e8c88700 1 -- 172.21.15.32:0/2404081877 shutdown_connections 2024-05-03T18:46:36.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:36.533+0000 7f99e8c88700 1 -- 172.21.15.32:0/2404081877 wait complete. 2024-05-03T18:46:36.563 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:36.534+0000 7f99e8c88700 1 Processor -- start 2024-05-03T18:46:36.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:36.534+0000 7f99e8c88700 1 -- start start 2024-05-03T18:46:36.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:36.534+0000 7f99e8c88700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f99e40756d0 0x7f99e419bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:36.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:36.534+0000 7f99e8c88700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99e4075ff0 0x7f99e419c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:36.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:36.534+0000 7f99e8c88700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f99e410fed0 0x7f99e41a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:36.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:36.534+0000 7f99e8c88700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f99e41158d0 con 0x7f99e4075ff0 2024-05-03T18:46:36.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:36.534+0000 7f99e8c88700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f99e4115750 con 0x7f99e410fed0 2024-05-03T18:46:36.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:36.534+0000 7f99e8c88700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f99e4115a50 con 0x7f99e40756d0 2024-05-03T18:46:36.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:36.534+0000 7f99e1d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99e4075ff0 0x7f99e419c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:36.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:36.534+0000 7f99e1d9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99e4075ff0 0x7f99e419c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46512/0 (socket says 172.21.15.32:46512) 2024-05-03T18:46:36.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:36.534+0000 7f99e1d9b700 1 -- 172.21.15.32:0/2644419857 learned_addr learned my addr 172.21.15.32:0/2644419857 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:46:36.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:36.534+0000 7f99e2d9d700 1 --2- 172.21.15.32:0/2644419857 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f99e410fed0 0x7f99e41a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:36.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:36.534+0000 7f99e259c700 1 --2- 172.21.15.32:0/2644419857 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f99e40756d0 0x7f99e419bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:36.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:36.534+0000 7f99e1d9b700 1 -- 172.21.15.32:0/2644419857 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f99e40756d0 msgr2=0x7f99e419bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:36.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:36.534+0000 7f99e1d9b700 1 --2- 172.21.15.32:0/2644419857 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f99e40756d0 0x7f99e419bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:36.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:36.534+0000 7f99e1d9b700 1 -- 172.21.15.32:0/2644419857 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f99e410fed0 msgr2=0x7f99e41a0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:36.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:36.534+0000 7f99e1d9b700 1 --2- 172.21.15.32:0/2644419857 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f99e410fed0 0x7f99e41a0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:36.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:36.534+0000 7f99e1d9b700 1 -- 172.21.15.32:0/2644419857 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99e41a0d60 con 0x7f99e4075ff0 2024-05-03T18:46:36.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:36.534+0000 7f99e2d9d700 1 --2- 172.21.15.32:0/2644419857 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f99e410fed0 0x7f99e41a0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T18:46:36.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:36.535+0000 7f99e1d9b700 1 --2- 172.21.15.32:0/2644419857 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99e4075ff0 0x7f99e419c2e0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f99cc00d980 tx=0x7f99cc00dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:46:36.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:36.535+0000 7f99db7fe700 1 -- 172.21.15.32:0/2644419857 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99cc017070 con 0x7f99e4075ff0 2024-05-03T18:46:36.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:36.535+0000 7f99db7fe700 1 -- 172.21.15.32:0/2644419857 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f99cc004830 con 0x7f99e4075ff0 2024-05-03T18:46:36.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:36.535+0000 7f99e8c88700 1 -- 172.21.15.32:0/2644419857 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99e41a1050 con 0x7f99e4075ff0 2024-05-03T18:46:36.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:36.535+0000 7f99db7fe700 1 -- 172.21.15.32:0/2644419857 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99cc005440 con 0x7f99e4075ff0 2024-05-03T18:46:36.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:36.535+0000 7f99e8c88700 1 -- 172.21.15.32:0/2644419857 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99e41a88d0 con 0x7f99e4075ff0 2024-05-03T18:46:36.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:36.536+0000 7f99e8c88700 1 -- 172.21.15.32:0/2644419857 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99e406a190 con 0x7f99e4075ff0 2024-05-03T18:46:36.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:36.538+0000 7f99db7fe700 1 -- 172.21.15.32:0/2644419857 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f99cc0055e0 con 0x7f99e4075ff0 2024-05-03T18:46:36.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:36.538+0000 7f99db7fe700 1 --2- 172.21.15.32:0/2644419857 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f99d00b2990 0x7f99d00b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:36.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:36.538+0000 7f99e259c700 1 --2- 172.21.15.32:0/2644419857 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f99d00b2990 0x7f99d00b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:36.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:36.538+0000 7f99db7fe700 1 -- 172.21.15.32:0/2644419857 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8269+0+0 (secure 0 0 0) 0x7f99cc0e2650 con 0x7f99e4075ff0 2024-05-03T18:46:36.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:36.539+0000 7f99e259c700 1 --2- 172.21.15.32:0/2644419857 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f99d00b2990 0x7f99d00b4e50 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f99dc009c10 tx=0x7f99dc009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:46:36.566 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:36.542+0000 7f99db7fe700 1 -- 172.21.15.32:0/2644419857 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f99cc020020 con 0x7f99e4075ff0 2024-05-03T18:46:36.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:36.761+0000 7f99e8c88700 1 -- 172.21.15.32:0/2644419857 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f99e402e510 con 0x7f99e4075ff0 2024-05-03T18:46:36.762 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:36.761+0000 7f99db7fe700 1 -- 172.21.15.32:0/2644419857 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f99cc0aa270 con 0x7f99e4075ff0 2024-05-03T18:46:36.762 INFO:teuthology.orchestra.run.smithi032.stdout:107374182476 2024-05-03T18:46:36.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:36.763+0000 7f99e8c88700 1 -- 172.21.15.32:0/2644419857 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f99d00b2990 msgr2=0x7f99d00b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:36.764 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:36.763+0000 7f99e8c88700 1 --2- 172.21.15.32:0/2644419857 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f99d00b2990 0x7f99d00b4e50 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f99dc009c10 tx=0x7f99dc009040 comp rx=0 tx=0).stop 2024-05-03T18:46:36.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:36.763+0000 7f99e8c88700 1 -- 172.21.15.32:0/2644419857 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99e4075ff0 msgr2=0x7f99e419c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:36.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:36.763+0000 7f99e8c88700 1 --2- 172.21.15.32:0/2644419857 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99e4075ff0 0x7f99e419c2e0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f99cc00d980 tx=0x7f99cc00dd40 comp rx=0 tx=0).stop 2024-05-03T18:46:36.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:36.763+0000 7f99e8c88700 1 -- 172.21.15.32:0/2644419857 shutdown_connections 2024-05-03T18:46:36.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:36.763+0000 7f99e8c88700 1 --2- 172.21.15.32:0/2644419857 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f99d00b2990 0x7f99d00b4e50 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:36.765 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:36.763+0000 7f99e8c88700 1 --2- 172.21.15.32:0/2644419857 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f99e410fed0 0x7f99e41a0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:36.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:36.763+0000 7f99e8c88700 1 --2- 172.21.15.32:0/2644419857 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f99e40756d0 0x7f99e419bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:36.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:36.763+0000 7f99e8c88700 1 --2- 172.21.15.32:0/2644419857 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99e4075ff0 0x7f99e419c2e0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:36.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:36.763+0000 7f99e8c88700 1 -- 172.21.15.32:0/2644419857 >> 172.21.15.32:0/2644419857 conn(0x7f99e407a650 msgr2=0x7f99e41006e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:46:36.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:36.763+0000 7f99e8c88700 1 -- 172.21.15.32:0/2644419857 shutdown_connections 2024-05-03T18:46:36.766 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:36.763+0000 7f99e8c88700 1 -- 172.21.15.32:0/2644419857 wait complete. 2024-05-03T18:46:37.028 INFO:tasks.cephadm.ceph_manager.ceph:need seq 253403070505 got 253403070509 for osd.9 2024-05-03T18:46:37.028 DEBUG:teuthology.parallel:result is None 2024-05-03T18:46:37.314 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:46:37 smithi032 ceph-mon[29763]: pgmap v335: 97 pgs: 97 active+clean; 631 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:46:37.314 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:46:37 smithi032 ceph-mon[29763]: from='client.? 172.21.15.32:0/2644419857' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-03T18:46:37.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:46:37 smithi045 ceph-mon[33299]: pgmap v335: 97 pgs: 97 active+clean; 631 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:46:37.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:46:37 smithi045 ceph-mon[33299]: from='client.? 172.21.15.32:0/2644419857' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-03T18:46:37.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:46:37 smithi113 ceph-mon[33540]: pgmap v335: 97 pgs: 97 active+clean; 631 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:46:37.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:46:37 smithi113 ceph-mon[33540]: from='client.? 172.21.15.32:0/2644419857' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-03T18:46:37.734 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182472 got 107374182476 for osd.3 2024-05-03T18:46:37.735 DEBUG:teuthology.parallel:result is None 2024-05-03T18:46:37.735 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-05-03T18:46:37.735 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid f500455e-097b-11ef-bc97-c7b262605968 -- ceph pg dump --format=json 2024-05-03T18:46:37.994 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:46:39.319 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:46:39 smithi032 ceph-mon[29763]: pgmap v336: 97 pgs: 97 active+clean; 631 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:46:39.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:46:39 smithi045 ceph-mon[33299]: pgmap v336: 97 pgs: 97 active+clean; 631 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:46:39.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:46:39 smithi113 ceph-mon[33540]: pgmap v336: 97 pgs: 97 active+clean; 631 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:46:39.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:39.549+0000 7f2ef40da700 1 -- 172.21.15.32:0/4280212895 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2eec075ff0 msgr2=0x7f2eec114160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:39.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:39.549+0000 7f2ef40da700 1 --2- 172.21.15.32:0/4280212895 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2eec075ff0 0x7f2eec114160 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f2edc009990 tx=0x7f2edc009ca0 comp rx=0 tx=0).stop 2024-05-03T18:46:39.553 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:39.549+0000 7f2ef40da700 1 -- 172.21.15.32:0/4280212895 shutdown_connections 2024-05-03T18:46:39.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:39.549+0000 7f2ef40da700 1 --2- 172.21.15.32:0/4280212895 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2eec1146a0 0x7f2eec116a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:39.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:39.549+0000 7f2ef40da700 1 --2- 172.21.15.32:0/4280212895 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2eec0756d0 0x7f2eec075ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:39.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:39.549+0000 7f2ef40da700 1 --2- 172.21.15.32:0/4280212895 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2eec075ff0 0x7f2eec114160 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:39.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:39.549+0000 7f2ef40da700 1 -- 172.21.15.32:0/4280212895 >> 172.21.15.32:0/4280212895 conn(0x7f2eec07a610 msgr2=0x7f2eec07aa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:46:39.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:39.549+0000 7f2ef40da700 1 -- 172.21.15.32:0/4280212895 shutdown_connections 2024-05-03T18:46:39.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:39.549+0000 7f2ef40da700 1 -- 172.21.15.32:0/4280212895 wait complete. 2024-05-03T18:46:39.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:39.550+0000 7f2ef40da700 1 Processor -- start 2024-05-03T18:46:39.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:39.550+0000 7f2ef40da700 1 -- start start 2024-05-03T18:46:39.554 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:39.550+0000 7f2ef40da700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2eec0756d0 0x7f2eec1a0170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:39.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:39.550+0000 7f2ef40da700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2eec075ff0 0x7f2eec1a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:39.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:39.550+0000 7f2ef40da700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2eec1146a0 0x7f2eec1a4a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:39.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:39.550+0000 7f2ef40da700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f2eec119d00 con 0x7f2eec075ff0 2024-05-03T18:46:39.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:39.550+0000 7f2ef40da700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2eec119b80 con 0x7f2eec0756d0 2024-05-03T18:46:39.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:39.550+0000 7f2ef40da700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f2eec119e80 con 0x7f2eec1146a0 2024-05-03T18:46:39.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:39.550+0000 7f2ef1675700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2eec075ff0 0x7f2eec1a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:39.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:39.550+0000 7f2ef1675700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2eec075ff0 0x7f2eec1a06b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46542/0 (socket says 172.21.15.32:46542) 2024-05-03T18:46:39.555 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:39.550+0000 7f2ef1675700 1 -- 172.21.15.32:0/597327690 learned_addr learned my addr 172.21.15.32:0/597327690 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:46:39.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:39.550+0000 7f2ef1e76700 1 --2- 172.21.15.32:0/597327690 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2eec0756d0 0x7f2eec1a0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:39.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:39.550+0000 7f2ef2677700 1 --2- 172.21.15.32:0/597327690 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2eec1146a0 0x7f2eec1a4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:39.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:39.550+0000 7f2ef1675700 1 -- 172.21.15.32:0/597327690 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2eec1146a0 msgr2=0x7f2eec1a4a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:39.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:39.550+0000 7f2ef1675700 1 --2- 172.21.15.32:0/597327690 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2eec1146a0 0x7f2eec1a4a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:39.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:39.550+0000 7f2ef1675700 1 -- 172.21.15.32:0/597327690 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2eec0756d0 msgr2=0x7f2eec1a0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:39.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:39.550+0000 7f2ef1675700 1 --2- 172.21.15.32:0/597327690 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2eec0756d0 0x7f2eec1a0170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:39.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:39.550+0000 7f2ef1675700 1 -- 172.21.15.32:0/597327690 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2eec1a5180 con 0x7f2eec075ff0 2024-05-03T18:46:39.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:39.551+0000 7f2ef1675700 1 --2- 172.21.15.32:0/597327690 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2eec075ff0 0x7f2eec1a06b0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f2edc000c00 tx=0x7f2edc0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:46:39.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:39.551+0000 7f2ee2ffd700 1 -- 172.21.15.32:0/597327690 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2edc027070 con 0x7f2eec075ff0 2024-05-03T18:46:39.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:39.551+0000 7f2ee2ffd700 1 -- 172.21.15.32:0/597327690 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2edc003fe0 con 0x7f2eec075ff0 2024-05-03T18:46:39.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:39.551+0000 7f2ef40da700 1 -- 172.21.15.32:0/597327690 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2eec1a5410 con 0x7f2eec075ff0 2024-05-03T18:46:39.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:39.551+0000 7f2ee2ffd700 1 -- 172.21.15.32:0/597327690 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2edc022e30 con 0x7f2eec075ff0 2024-05-03T18:46:39.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:39.551+0000 7f2ef40da700 1 -- 172.21.15.32:0/597327690 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2eec1accf0 con 0x7f2eec075ff0 2024-05-03T18:46:39.556 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:39.552+0000 7f2ef40da700 1 -- 172.21.15.32:0/597327690 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2eec06a190 con 0x7f2eec075ff0 2024-05-03T18:46:39.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:39.553+0000 7f2ee2ffd700 1 -- 172.21.15.32:0/597327690 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f2edc02a070 con 0x7f2eec075ff0 2024-05-03T18:46:39.557 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:39.554+0000 7f2ee2ffd700 1 --2- 172.21.15.32:0/597327690 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f2ed80b2910 0x7f2ed80b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:39.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:39.554+0000 7f2ef1e76700 1 --2- 172.21.15.32:0/597327690 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f2ed80b2910 0x7f2ed80b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:39.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:39.554+0000 7f2ee2ffd700 1 -- 172.21.15.32:0/597327690 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8269+0+0 (secure 0 0 0) 0x7f2edc0e3300 con 0x7f2eec075ff0 2024-05-03T18:46:39.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:39.554+0000 7f2ef1e76700 1 --2- 172.21.15.32:0/597327690 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f2ed80b2910 0x7f2ed80b4dd0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f2ee800ac50 tx=0x7f2ee800a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:46:39.558 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:39.557+0000 7f2ee2ffd700 1 -- 172.21.15.32:0/597327690 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f2edc013070 con 0x7f2eec075ff0 2024-05-03T18:46:39.768 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:39.767+0000 7f2ef40da700 1 -- 172.21.15.32:0/597327690 --> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2eec113b70 con 0x7f2ed80b2910 2024-05-03T18:46:39.776 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:39.775+0000 7f2ee2ffd700 1 -- 172.21.15.32:0/597327690 <== mgr.14152 v2:172.21.15.32:6800/1796547904 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258703 (secure 0 0 0) 0x7f2eec113b70 con 0x7f2ed80b2910 2024-05-03T18:46:39.780 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-03T18:46:39.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:39.778+0000 7f2ef40da700 1 -- 172.21.15.32:0/597327690 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f2ed80b2910 msgr2=0x7f2ed80b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:39.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:39.778+0000 7f2ef40da700 1 --2- 172.21.15.32:0/597327690 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f2ed80b2910 0x7f2ed80b4dd0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f2ee800ac50 tx=0x7f2ee800a380 comp rx=0 tx=0).stop 2024-05-03T18:46:39.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:39.778+0000 7f2ef40da700 1 -- 172.21.15.32:0/597327690 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2eec075ff0 msgr2=0x7f2eec1a06b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:39.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:39.778+0000 7f2ef40da700 1 --2- 172.21.15.32:0/597327690 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2eec075ff0 0x7f2eec1a06b0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f2edc000c00 tx=0x7f2edc0047b0 comp rx=0 tx=0).stop 2024-05-03T18:46:39.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:39.778+0000 7f2ef40da700 1 -- 172.21.15.32:0/597327690 shutdown_connections 2024-05-03T18:46:39.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:39.778+0000 7f2ef40da700 1 --2- 172.21.15.32:0/597327690 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f2ed80b2910 0x7f2ed80b4dd0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:39.782 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:39.778+0000 7f2ef40da700 1 --2- 172.21.15.32:0/597327690 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2eec1146a0 0x7f2eec1a4a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:39.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:39.778+0000 7f2ef40da700 1 --2- 172.21.15.32:0/597327690 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2eec0756d0 0x7f2eec1a0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:39.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:39.778+0000 7f2ef40da700 1 --2- 172.21.15.32:0/597327690 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2eec075ff0 0x7f2eec1a06b0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:39.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:39.778+0000 7f2ef40da700 1 -- 172.21.15.32:0/597327690 >> 172.21.15.32:0/597327690 conn(0x7f2eec07a610 msgr2=0x7f2eec113460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:46:39.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:39.778+0000 7f2ef40da700 1 -- 172.21.15.32:0/597327690 shutdown_connections 2024-05-03T18:46:39.783 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:39.778+0000 7f2ef40da700 1 -- 172.21.15.32:0/597327690 wait complete. 2024-05-03T18:46:39.783 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-05-03T18:46:40.558 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":336,"stamp":"2024-05-03T18:46:37.991333+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":646436,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":653,"num_read_kb":411,"num_write":284,"num_write_kb":2253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":2951957,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":230,"ondisk_log_size":230,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":374024,"kb_used_data":12324,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124567800,"statfs":{"total":1151940427776,"available":1151557427200,"internally_reserved":0,"allocated":12619776,"data_stored":7036068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":370193723},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":80,"num_read_kb":40,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003562"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.585847+0000","last_change":"2024-05-03T18:44:49.430467+0000","last_active":"2024-05-03T18:45:47.585847+0000","last_peered":"2024-05-03T18:45:47.585847+0000","last_clean":"2024-05-03T18:45:47.585847+0000","last_became_active":"2024-05-03T18:44:49.430147+0000","last_became_peered":"2024-05-03T18:44:49.430147+0000","last_unstale":"2024-05-03T18:45:47.585847+0000","last_undegraded":"2024-05-03T18:45:47.585847+0000","last_fullsized":"2024-05-03T18:45:47.585847+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:49:22.725690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.586166+0000","last_change":"2024-05-03T18:44:49.432057+0000","last_active":"2024-05-03T18:45:47.586166+0000","last_peered":"2024-05-03T18:45:47.586166+0000","last_clean":"2024-05-03T18:45:47.586166+0000","last_became_active":"2024-05-03T18:44:49.430867+0000","last_became_peered":"2024-05-03T18:44:49.430867+0000","last_unstale":"2024-05-03T18:45:47.586166+0000","last_undegraded":"2024-05-03T18:45:47.586166+0000","last_fullsized":"2024-05-03T18:45:47.586166+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:31:16.470041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.586095+0000","last_change":"2024-05-03T18:44:49.433622+0000","last_active":"2024-05-03T18:45:47.586095+0000","last_peered":"2024-05-03T18:45:47.586095+0000","last_clean":"2024-05-03T18:45:47.586095+0000","last_became_active":"2024-05-03T18:44:49.431803+0000","last_became_peered":"2024-05-03T18:44:49.431803+0000","last_unstale":"2024-05-03T18:45:47.586095+0000","last_undegraded":"2024-05-03T18:45:47.586095+0000","last_fullsized":"2024-05-03T18:45:47.586095+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:40:18.492498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.635138+0000","last_change":"2024-05-03T18:44:49.437469+0000","last_active":"2024-05-03T18:45:47.635138+0000","last_peered":"2024-05-03T18:45:47.635138+0000","last_clean":"2024-05-03T18:45:47.635138+0000","last_became_active":"2024-05-03T18:44:49.435519+0000","last_became_peered":"2024-05-03T18:44:49.435519+0000","last_unstale":"2024-05-03T18:45:47.635138+0000","last_undegraded":"2024-05-03T18:45:47.635138+0000","last_fullsized":"2024-05-03T18:45:47.635138+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:11:07.246146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.958037+0000","last_change":"2024-05-03T18:45:47.958037+0000","last_active":"2024-05-03T18:45:47.958037+0000","last_peered":"2024-05-03T18:45:47.958037+0000","last_clean":"2024-05-03T18:45:47.958037+0000","last_became_active":"2024-05-03T18:45:47.957558+0000","last_became_peered":"2024-05-03T18:45:47.957558+0000","last_unstale":"2024-05-03T18:45:47.958037+0000","last_undegraded":"2024-05-03T18:45:47.958037+0000","last_fullsized":"2024-05-03T18:45:47.958037+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:19:31.077969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.589949+0000","last_change":"2024-05-03T18:45:47.589949+0000","last_active":"2024-05-03T18:45:47.589949+0000","last_peered":"2024-05-03T18:45:47.589949+0000","last_clean":"2024-05-03T18:45:47.589949+0000","last_became_active":"2024-05-03T18:45:47.589517+0000","last_became_peered":"2024-05-03T18:45:47.589517+0000","last_unstale":"2024-05-03T18:45:47.589949+0000","last_undegraded":"2024-05-03T18:45:47.589949+0000","last_fullsized":"2024-05-03T18:45:47.589949+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:00:41.296563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.589866+0000","last_change":"2024-05-03T18:45:47.589866+0000","last_active":"2024-05-03T18:45:47.589866+0000","last_peered":"2024-05-03T18:45:47.589866+0000","last_clean":"2024-05-03T18:45:47.589866+0000","last_became_active":"2024-05-03T18:45:47.589402+0000","last_became_peered":"2024-05-03T18:45:47.589402+0000","last_unstale":"2024-05-03T18:45:47.589866+0000","last_undegraded":"2024-05-03T18:45:47.589866+0000","last_fullsized":"2024-05-03T18:45:47.589866+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:55:48.047379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,8],"acting":[7,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.582342+0000","last_change":"2024-05-03T18:44:49.432804+0000","last_active":"2024-05-03T18:45:47.582342+0000","last_peered":"2024-05-03T18:45:47.582342+0000","last_clean":"2024-05-03T18:45:47.582342+0000","last_became_active":"2024-05-03T18:44:49.431828+0000","last_became_peered":"2024-05-03T18:44:49.431828+0000","last_unstale":"2024-05-03T18:45:47.582342+0000","last_undegraded":"2024-05-03T18:45:47.582342+0000","last_fullsized":"2024-05-03T18:45:47.582342+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:06:40.810972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.583535+0000","last_change":"2024-05-03T18:44:49.428919+0000","last_active":"2024-05-03T18:45:47.583535+0000","last_peered":"2024-05-03T18:45:47.583535+0000","last_clean":"2024-05-03T18:45:47.583535+0000","last_became_active":"2024-05-03T18:44:49.428350+0000","last_became_peered":"2024-05-03T18:44:49.428350+0000","last_unstale":"2024-05-03T18:45:47.583535+0000","last_undegraded":"2024-05-03T18:45:47.583535+0000","last_fullsized":"2024-05-03T18:45:47.583535+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T18:50:09.895485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.587062+0000","last_change":"2024-05-03T18:44:49.431765+0000","last_active":"2024-05-03T18:45:47.587062+0000","last_peered":"2024-05-03T18:45:47.587062+0000","last_clean":"2024-05-03T18:45:47.587062+0000","last_became_active":"2024-05-03T18:44:49.431417+0000","last_became_peered":"2024-05-03T18:44:49.431417+0000","last_unstale":"2024-05-03T18:45:47.587062+0000","last_undegraded":"2024-05-03T18:45:47.587062+0000","last_fullsized":"2024-05-03T18:45:47.587062+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:24:58.097954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.584833+0000","last_change":"2024-05-03T18:44:49.434154+0000","last_active":"2024-05-03T18:45:47.584833+0000","last_peered":"2024-05-03T18:45:47.584833+0000","last_clean":"2024-05-03T18:45:47.584833+0000","last_became_active":"2024-05-03T18:44:49.433806+0000","last_became_peered":"2024-05-03T18:44:49.433806+0000","last_unstale":"2024-05-03T18:45:47.584833+0000","last_undegraded":"2024-05-03T18:45:47.584833+0000","last_fullsized":"2024-05-03T18:45:47.584833+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:52:39.233556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.635100+0000","last_change":"2024-05-03T18:44:49.435385+0000","last_active":"2024-05-03T18:45:47.635100+0000","last_peered":"2024-05-03T18:45:47.635100+0000","last_clean":"2024-05-03T18:45:47.635100+0000","last_became_active":"2024-05-03T18:44:49.434727+0000","last_became_peered":"2024-05-03T18:44:49.434727+0000","last_unstale":"2024-05-03T18:45:47.635100+0000","last_undegraded":"2024-05-03T18:45:47.635100+0000","last_fullsized":"2024-05-03T18:45:47.635100+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:08:45.986371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.956820+0000","last_change":"2024-05-03T18:44:49.428536+0000","last_active":"2024-05-03T18:45:47.956820+0000","last_peered":"2024-05-03T18:45:47.956820+0000","last_clean":"2024-05-03T18:45:47.956820+0000","last_became_active":"2024-05-03T18:44:49.427630+0000","last_became_peered":"2024-05-03T18:44:49.427630+0000","last_unstale":"2024-05-03T18:45:47.956820+0000","last_undegraded":"2024-05-03T18:45:47.956820+0000","last_fullsized":"2024-05-03T18:45:47.956820+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:24:09.622403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.585815+0000","last_change":"2024-05-03T18:44:49.429236+0000","last_active":"2024-05-03T18:45:47.585815+0000","last_peered":"2024-05-03T18:45:47.585815+0000","last_clean":"2024-05-03T18:45:47.585815+0000","last_became_active":"2024-05-03T18:44:49.428942+0000","last_became_peered":"2024-05-03T18:44:49.428942+0000","last_unstale":"2024-05-03T18:45:47.585815+0000","last_undegraded":"2024-05-03T18:45:47.585815+0000","last_fullsized":"2024-05-03T18:45:47.585815+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:16:44.241736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.585732+0000","last_change":"2024-05-03T18:44:49.433388+0000","last_active":"2024-05-03T18:45:47.585732+0000","last_peered":"2024-05-03T18:45:47.585732+0000","last_clean":"2024-05-03T18:45:47.585732+0000","last_became_active":"2024-05-03T18:44:49.431869+0000","last_became_peered":"2024-05-03T18:44:49.431869+0000","last_unstale":"2024-05-03T18:45:47.585732+0000","last_undegraded":"2024-05-03T18:45:47.585732+0000","last_fullsized":"2024-05-03T18:45:47.585732+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:36:52.803391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.582873+0000","last_change":"2024-05-03T18:44:49.425781+0000","last_active":"2024-05-03T18:45:47.582873+0000","last_peered":"2024-05-03T18:45:47.582873+0000","last_clean":"2024-05-03T18:45:47.582873+0000","last_became_active":"2024-05-03T18:44:49.425248+0000","last_became_peered":"2024-05-03T18:44:49.425248+0000","last_unstale":"2024-05-03T18:45:47.582873+0000","last_undegraded":"2024-05-03T18:45:47.582873+0000","last_fullsized":"2024-05-03T18:45:47.582873+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:27:25.017459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.589399+0000","last_change":"2024-05-03T18:45:47.589399+0000","last_active":"2024-05-03T18:45:47.589399+0000","last_peered":"2024-05-03T18:45:47.589399+0000","last_clean":"2024-05-03T18:45:47.589399+0000","last_became_active":"2024-05-03T18:45:47.589028+0000","last_became_peered":"2024-05-03T18:45:47.589028+0000","last_unstale":"2024-05-03T18:45:47.589399+0000","last_undegraded":"2024-05-03T18:45:47.589399+0000","last_fullsized":"2024-05-03T18:45:47.589399+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:00:47.913452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,0],"acting":[11,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.584936+0000","last_change":"2024-05-03T18:44:49.427176+0000","last_active":"2024-05-03T18:45:47.584936+0000","last_peered":"2024-05-03T18:45:47.584936+0000","last_clean":"2024-05-03T18:45:47.584936+0000","last_became_active":"2024-05-03T18:44:49.426546+0000","last_became_peered":"2024-05-03T18:44:49.426546+0000","last_unstale":"2024-05-03T18:45:47.584936+0000","last_undegraded":"2024-05-03T18:45:47.584936+0000","last_fullsized":"2024-05-03T18:45:47.584936+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:29:22.343376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.589214+0000","last_change":"2024-05-03T18:44:49.435037+0000","last_active":"2024-05-03T18:45:47.589214+0000","last_peered":"2024-05-03T18:45:47.589214+0000","last_clean":"2024-05-03T18:45:47.589214+0000","last_became_active":"2024-05-03T18:44:49.434693+0000","last_became_peered":"2024-05-03T18:44:49.434693+0000","last_unstale":"2024-05-03T18:45:47.589214+0000","last_undegraded":"2024-05-03T18:45:47.589214+0000","last_fullsized":"2024-05-03T18:45:47.589214+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:26:33.923608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"78'9","reported_seq":35,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.585020+0000","last_change":"2024-05-03T18:44:46.402606+0000","last_active":"2024-05-03T18:45:47.585020+0000","last_peered":"2024-05-03T18:45:47.585020+0000","last_clean":"2024-05-03T18:45:47.585020+0000","last_became_active":"2024-05-03T18:44:46.402245+0000","last_became_peered":"2024-05-03T18:44:46.402245+0000","last_unstale":"2024-05-03T18:45:47.585020+0000","last_undegraded":"2024-05-03T18:45:47.585020+0000","last_fullsized":"2024-05-03T18:45:47.585020+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_clean_scrub_stamp":"2024-05-03T18:44:45.380544+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:29:33.726119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.582919+0000","last_change":"2024-05-03T18:44:49.435279+0000","last_active":"2024-05-03T18:45:47.582919+0000","last_peered":"2024-05-03T18:45:47.582919+0000","last_clean":"2024-05-03T18:45:47.582919+0000","last_became_active":"2024-05-03T18:44:49.434779+0000","last_became_peered":"2024-05-03T18:44:49.434779+0000","last_unstale":"2024-05-03T18:45:47.582919+0000","last_undegraded":"2024-05-03T18:45:47.582919+0000","last_fullsized":"2024-05-03T18:45:47.582919+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:56:56.762051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"78'2","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.582807+0000","last_change":"2024-05-03T18:44:46.644899+0000","last_active":"2024-05-03T18:45:47.582807+0000","last_peered":"2024-05-03T18:45:47.582807+0000","last_clean":"2024-05-03T18:45:47.582807+0000","last_became_active":"2024-05-03T18:44:46.644508+0000","last_became_peered":"2024-05-03T18:44:46.644508+0000","last_unstale":"2024-05-03T18:45:47.582807+0000","last_undegraded":"2024-05-03T18:45:47.582807+0000","last_fullsized":"2024-05-03T18:45:47.582807+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_clean_scrub_stamp":"2024-05-03T18:44:45.380544+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T06:38:09.379634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.586375+0000","last_change":"2024-05-03T18:44:49.429720+0000","last_active":"2024-05-03T18:45:47.586375+0000","last_peered":"2024-05-03T18:45:47.586375+0000","last_clean":"2024-05-03T18:45:47.586375+0000","last_became_active":"2024-05-03T18:44:49.428399+0000","last_became_peered":"2024-05-03T18:44:49.428399+0000","last_unstale":"2024-05-03T18:45:47.586375+0000","last_undegraded":"2024-05-03T18:45:47.586375+0000","last_fullsized":"2024-05-03T18:45:47.586375+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:17:57.325926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"78'4","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.585620+0000","last_change":"2024-05-03T18:44:46.410397+0000","last_active":"2024-05-03T18:45:47.585620+0000","last_peered":"2024-05-03T18:45:47.585620+0000","last_clean":"2024-05-03T18:45:47.585620+0000","last_became_active":"2024-05-03T18:44:46.410030+0000","last_became_peered":"2024-05-03T18:44:46.410030+0000","last_unstale":"2024-05-03T18:45:47.585620+0000","last_undegraded":"2024-05-03T18:45:47.585620+0000","last_fullsized":"2024-05-03T18:45:47.585620+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_clean_scrub_stamp":"2024-05-03T18:44:45.380544+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:15:26.130070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.583088+0000","last_change":"2024-05-03T18:44:49.430335+0000","last_active":"2024-05-03T18:45:47.583088+0000","last_peered":"2024-05-03T18:45:47.583088+0000","last_clean":"2024-05-03T18:45:47.583088+0000","last_became_active":"2024-05-03T18:44:49.429991+0000","last_became_peered":"2024-05-03T18:44:49.429991+0000","last_unstale":"2024-05-03T18:45:47.583088+0000","last_undegraded":"2024-05-03T18:45:47.583088+0000","last_fullsized":"2024-05-03T18:45:47.583088+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:06:28.718138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"78'2","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.585877+0000","last_change":"2024-05-03T18:44:46.406552+0000","last_active":"2024-05-03T18:45:47.585877+0000","last_peered":"2024-05-03T18:45:47.585877+0000","last_clean":"2024-05-03T18:45:47.585877+0000","last_became_active":"2024-05-03T18:44:46.406160+0000","last_became_peered":"2024-05-03T18:44:46.406160+0000","last_unstale":"2024-05-03T18:45:47.585877+0000","last_undegraded":"2024-05-03T18:45:47.585877+0000","last_fullsized":"2024-05-03T18:45:47.585877+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_clean_scrub_stamp":"2024-05-03T18:44:45.380544+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:53:13.769225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.584609+0000","last_change":"2024-05-03T18:44:49.441301+0000","last_active":"2024-05-03T18:45:47.584609+0000","last_peered":"2024-05-03T18:45:47.584609+0000","last_clean":"2024-05-03T18:45:47.584609+0000","last_became_active":"2024-05-03T18:44:49.440936+0000","last_became_peered":"2024-05-03T18:44:49.440936+0000","last_unstale":"2024-05-03T18:45:47.584609+0000","last_undegraded":"2024-05-03T18:45:47.584609+0000","last_fullsized":"2024-05-03T18:45:47.584609+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:55:41.257325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"78'6","reported_seq":125,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:46:34.616606+0000","last_change":"2024-05-03T18:44:46.404827+0000","last_active":"2024-05-03T18:46:34.616606+0000","last_peered":"2024-05-03T18:46:34.616606+0000","last_clean":"2024-05-03T18:46:34.616606+0000","last_became_active":"2024-05-03T18:44:46.404455+0000","last_became_peered":"2024-05-03T18:44:46.404455+0000","last_unstale":"2024-05-03T18:46:34.616606+0000","last_undegraded":"2024-05-03T18:46:34.616606+0000","last_fullsized":"2024-05-03T18:46:34.616606+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_clean_scrub_stamp":"2024-05-03T18:44:45.380544+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:23:32.855160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":94,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.585568+0000","last_change":"2024-05-03T18:44:49.433656+0000","last_active":"2024-05-03T18:45:47.585568+0000","last_peered":"2024-05-03T18:45:47.585568+0000","last_clean":"2024-05-03T18:45:47.585568+0000","last_became_active":"2024-05-03T18:44:49.433326+0000","last_became_peered":"2024-05-03T18:44:49.433326+0000","last_unstale":"2024-05-03T18:45:47.585568+0000","last_undegraded":"2024-05-03T18:45:47.585568+0000","last_fullsized":"2024-05-03T18:45:47.585568+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:44:56.937334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"78'3","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.586938+0000","last_change":"2024-05-03T18:44:46.403960+0000","last_active":"2024-05-03T18:45:47.586938+0000","last_peered":"2024-05-03T18:45:47.586938+0000","last_clean":"2024-05-03T18:45:47.586938+0000","last_became_active":"2024-05-03T18:44:46.403560+0000","last_became_peered":"2024-05-03T18:44:46.403560+0000","last_unstale":"2024-05-03T18:45:47.586938+0000","last_undegraded":"2024-05-03T18:45:47.586938+0000","last_fullsized":"2024-05-03T18:45:47.586938+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_clean_scrub_stamp":"2024-05-03T18:44:45.380544+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:58:32.823640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.957785+0000","last_change":"2024-05-03T18:45:47.957785+0000","last_active":"2024-05-03T18:45:47.957785+0000","last_peered":"2024-05-03T18:45:47.957785+0000","last_clean":"2024-05-03T18:45:47.957785+0000","last_became_active":"2024-05-03T18:45:47.957429+0000","last_became_peered":"2024-05-03T18:45:47.957429+0000","last_unstale":"2024-05-03T18:45:47.957785+0000","last_undegraded":"2024-05-03T18:45:47.957785+0000","last_fullsized":"2024-05-03T18:45:47.957785+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T18:47:27.922032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,3],"acting":[5,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"78'2","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.956500+0000","last_change":"2024-05-03T18:44:46.400282+0000","last_active":"2024-05-03T18:45:47.956500+0000","last_peered":"2024-05-03T18:45:47.956500+0000","last_clean":"2024-05-03T18:45:47.956500+0000","last_became_active":"2024-05-03T18:44:46.399873+0000","last_became_peered":"2024-05-03T18:44:46.399873+0000","last_unstale":"2024-05-03T18:45:47.956500+0000","last_undegraded":"2024-05-03T18:45:47.956500+0000","last_fullsized":"2024-05-03T18:45:47.956500+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_clean_scrub_stamp":"2024-05-03T18:44:45.380544+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:12:10.005145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.582371+0000","last_change":"2024-05-03T18:44:49.422311+0000","last_active":"2024-05-03T18:45:47.582371+0000","last_peered":"2024-05-03T18:45:47.582371+0000","last_clean":"2024-05-03T18:45:47.582371+0000","last_became_active":"2024-05-03T18:44:49.421962+0000","last_became_peered":"2024-05-03T18:44:49.421962+0000","last_unstale":"2024-05-03T18:45:47.582371+0000","last_undegraded":"2024-05-03T18:45:47.582371+0000","last_fullsized":"2024-05-03T18:45:47.582371+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:50:46.407989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"78'1","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.583898+0000","last_change":"2024-05-03T18:44:46.410969+0000","last_active":"2024-05-03T18:45:47.583898+0000","last_peered":"2024-05-03T18:45:47.583898+0000","last_clean":"2024-05-03T18:45:47.583898+0000","last_became_active":"2024-05-03T18:44:46.410089+0000","last_became_peered":"2024-05-03T18:44:46.410089+0000","last_unstale":"2024-05-03T18:45:47.583898+0000","last_undegraded":"2024-05-03T18:45:47.583898+0000","last_fullsized":"2024-05-03T18:45:47.583898+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_clean_scrub_stamp":"2024-05-03T18:44:45.380544+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:13:51.641235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.584266+0000","last_change":"2024-05-03T18:44:49.426946+0000","last_active":"2024-05-03T18:45:47.584266+0000","last_peered":"2024-05-03T18:45:47.584266+0000","last_clean":"2024-05-03T18:45:47.584266+0000","last_became_active":"2024-05-03T18:44:49.426609+0000","last_became_peered":"2024-05-03T18:44:49.426609+0000","last_unstale":"2024-05-03T18:45:47.584266+0000","last_undegraded":"2024-05-03T18:45:47.584266+0000","last_fullsized":"2024-05-03T18:45:47.584266+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:34:59.944769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"78'3","reported_seq":120,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:46:35.617295+0000","last_change":"2024-05-03T18:44:46.405415+0000","last_active":"2024-05-03T18:46:35.617295+0000","last_peered":"2024-05-03T18:46:35.617295+0000","last_clean":"2024-05-03T18:46:35.617295+0000","last_became_active":"2024-05-03T18:44:46.404928+0000","last_became_peered":"2024-05-03T18:44:46.404928+0000","last_unstale":"2024-05-03T18:46:35.617295+0000","last_undegraded":"2024-05-03T18:46:35.617295+0000","last_fullsized":"2024-05-03T18:46:35.617295+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_clean_scrub_stamp":"2024-05-03T18:44:45.380544+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:33:10.979432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":92,"num_read_kb":92,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.583489+0000","last_change":"2024-05-03T18:44:49.427873+0000","last_active":"2024-05-03T18:45:47.583489+0000","last_peered":"2024-05-03T18:45:47.583489+0000","last_clean":"2024-05-03T18:45:47.583489+0000","last_became_active":"2024-05-03T18:44:49.426797+0000","last_became_peered":"2024-05-03T18:44:49.426797+0000","last_unstale":"2024-05-03T18:45:47.583489+0000","last_undegraded":"2024-05-03T18:45:47.583489+0000","last_fullsized":"2024-05-03T18:45:47.583489+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:07:28.989454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"78'2","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.584594+0000","last_change":"2024-05-03T18:44:46.407167+0000","last_active":"2024-05-03T18:45:47.584594+0000","last_peered":"2024-05-03T18:45:47.584594+0000","last_clean":"2024-05-03T18:45:47.584594+0000","last_became_active":"2024-05-03T18:44:46.406787+0000","last_became_peered":"2024-05-03T18:44:46.406787+0000","last_unstale":"2024-05-03T18:45:47.584594+0000","last_undegraded":"2024-05-03T18:45:47.584594+0000","last_fullsized":"2024-05-03T18:45:47.584594+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_clean_scrub_stamp":"2024-05-03T18:44:45.380544+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:33:12.854056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.957910+0000","last_change":"2024-05-03T18:45:47.957910+0000","last_active":"2024-05-03T18:45:47.957910+0000","last_peered":"2024-05-03T18:45:47.957910+0000","last_clean":"2024-05-03T18:45:47.957910+0000","last_became_active":"2024-05-03T18:45:47.957452+0000","last_became_peered":"2024-05-03T18:45:47.957452+0000","last_unstale":"2024-05-03T18:45:47.957910+0000","last_undegraded":"2024-05-03T18:45:47.957910+0000","last_fullsized":"2024-05-03T18:45:47.957910+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:22:03.503320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"78'4","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.584723+0000","last_change":"2024-05-03T18:44:46.647273+0000","last_active":"2024-05-03T18:45:47.584723+0000","last_peered":"2024-05-03T18:45:47.584723+0000","last_clean":"2024-05-03T18:45:47.584723+0000","last_became_active":"2024-05-03T18:44:46.646881+0000","last_became_peered":"2024-05-03T18:44:46.646881+0000","last_unstale":"2024-05-03T18:45:47.584723+0000","last_undegraded":"2024-05-03T18:45:47.584723+0000","last_fullsized":"2024-05-03T18:45:47.584723+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_clean_scrub_stamp":"2024-05-03T18:44:45.380544+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:03:47.988180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.583137+0000","last_change":"2024-05-03T18:44:49.436525+0000","last_active":"2024-05-03T18:45:47.583137+0000","last_peered":"2024-05-03T18:45:47.583137+0000","last_clean":"2024-05-03T18:45:47.583137+0000","last_became_active":"2024-05-03T18:44:49.436109+0000","last_became_peered":"2024-05-03T18:44:49.436109+0000","last_unstale":"2024-05-03T18:45:47.583137+0000","last_undegraded":"2024-05-03T18:45:47.583137+0000","last_fullsized":"2024-05-03T18:45:47.583137+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:08:42.239248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"78'27","reported_seq":244,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:46:36.618582+0000","last_change":"2024-05-03T18:44:46.407936+0000","last_active":"2024-05-03T18:46:36.618582+0000","last_peered":"2024-05-03T18:46:36.618582+0000","last_clean":"2024-05-03T18:46:36.618582+0000","last_became_active":"2024-05-03T18:44:46.407504+0000","last_became_peered":"2024-05-03T18:44:46.407504+0000","last_unstale":"2024-05-03T18:46:36.618582+0000","last_undegraded":"2024-05-03T18:46:36.618582+0000","last_fullsized":"2024-05-03T18:46:36.618582+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_clean_scrub_stamp":"2024-05-03T18:44:45.380544+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:26:27.213572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":51833,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":192,"num_read_kb":146,"num_write":28,"num_write_kb":60,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.585324+0000","last_change":"2024-05-03T18:44:49.438509+0000","last_active":"2024-05-03T18:45:47.585324+0000","last_peered":"2024-05-03T18:45:47.585324+0000","last_clean":"2024-05-03T18:45:47.585324+0000","last_became_active":"2024-05-03T18:44:49.438185+0000","last_became_peered":"2024-05-03T18:44:49.438185+0000","last_unstale":"2024-05-03T18:45:47.585324+0000","last_undegraded":"2024-05-03T18:45:47.585324+0000","last_fullsized":"2024-05-03T18:45:47.585324+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:57:28.072095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"78'6","reported_seq":33,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.589344+0000","last_change":"2024-05-03T18:44:46.407842+0000","last_active":"2024-05-03T18:45:47.589344+0000","last_peered":"2024-05-03T18:45:47.589344+0000","last_clean":"2024-05-03T18:45:47.589344+0000","last_became_active":"2024-05-03T18:44:46.407506+0000","last_became_peered":"2024-05-03T18:44:46.407506+0000","last_unstale":"2024-05-03T18:45:47.589344+0000","last_undegraded":"2024-05-03T18:45:47.589344+0000","last_fullsized":"2024-05-03T18:45:47.589344+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_clean_scrub_stamp":"2024-05-03T18:44:45.380544+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:51:33.101539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.956231+0000","last_change":"2024-05-03T18:44:49.433214+0000","last_active":"2024-05-03T18:45:47.956231+0000","last_peered":"2024-05-03T18:45:47.956231+0000","last_clean":"2024-05-03T18:45:47.956231+0000","last_became_active":"2024-05-03T18:44:49.432879+0000","last_became_peered":"2024-05-03T18:44:49.432879+0000","last_unstale":"2024-05-03T18:45:47.956231+0000","last_undegraded":"2024-05-03T18:45:47.956231+0000","last_fullsized":"2024-05-03T18:45:47.956231+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:35:17.920304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"78'3","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.583880+0000","last_change":"2024-05-03T18:44:46.405280+0000","last_active":"2024-05-03T18:45:47.583880+0000","last_peered":"2024-05-03T18:45:47.583880+0000","last_clean":"2024-05-03T18:45:47.583880+0000","last_became_active":"2024-05-03T18:44:46.404126+0000","last_became_peered":"2024-05-03T18:44:46.404126+0000","last_unstale":"2024-05-03T18:45:47.583880+0000","last_undegraded":"2024-05-03T18:45:47.583880+0000","last_fullsized":"2024-05-03T18:45:47.583880+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_clean_scrub_stamp":"2024-05-03T18:44:45.380544+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:14:52.146440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.585033+0000","last_change":"2024-05-03T18:44:49.430582+0000","last_active":"2024-05-03T18:45:47.585033+0000","last_peered":"2024-05-03T18:45:47.585033+0000","last_clean":"2024-05-03T18:45:47.585033+0000","last_became_active":"2024-05-03T18:44:49.430246+0000","last_became_peered":"2024-05-03T18:44:49.430246+0000","last_unstale":"2024-05-03T18:45:47.585033+0000","last_undegraded":"2024-05-03T18:45:47.585033+0000","last_fullsized":"2024-05-03T18:45:47.585033+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:17:13.504586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"78'3","reported_seq":123,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:46:36.618681+0000","last_change":"2024-05-03T18:44:46.404276+0000","last_active":"2024-05-03T18:46:36.618681+0000","last_peered":"2024-05-03T18:46:36.618681+0000","last_clean":"2024-05-03T18:46:36.618681+0000","last_became_active":"2024-05-03T18:44:46.403843+0000","last_became_peered":"2024-05-03T18:44:46.403843+0000","last_unstale":"2024-05-03T18:46:36.618681+0000","last_undegraded":"2024-05-03T18:46:36.618681+0000","last_fullsized":"2024-05-03T18:46:36.618681+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_clean_scrub_stamp":"2024-05-03T18:44:45.380544+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:05:49.483444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.584250+0000","last_change":"2024-05-03T18:44:49.427950+0000","last_active":"2024-05-03T18:45:47.584250+0000","last_peered":"2024-05-03T18:45:47.584250+0000","last_clean":"2024-05-03T18:45:47.584250+0000","last_became_active":"2024-05-03T18:44:49.427469+0000","last_became_peered":"2024-05-03T18:44:49.427469+0000","last_unstale":"2024-05-03T18:45:47.584250+0000","last_undegraded":"2024-05-03T18:45:47.584250+0000","last_fullsized":"2024-05-03T18:45:47.584250+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:33:35.690809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"78'15","reported_seq":41,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.585236+0000","last_change":"2024-05-03T18:44:46.645189+0000","last_active":"2024-05-03T18:45:47.585236+0000","last_peered":"2024-05-03T18:45:47.585236+0000","last_clean":"2024-05-03T18:45:47.585236+0000","last_became_active":"2024-05-03T18:44:46.644642+0000","last_became_peered":"2024-05-03T18:44:46.644642+0000","last_unstale":"2024-05-03T18:45:47.585236+0000","last_undegraded":"2024-05-03T18:45:47.585236+0000","last_fullsized":"2024-05-03T18:45:47.585236+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_clean_scrub_stamp":"2024-05-03T18:44:45.380544+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:16:49.003568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.584408+0000","last_change":"2024-05-03T18:44:49.438931+0000","last_active":"2024-05-03T18:45:47.584408+0000","last_peered":"2024-05-03T18:45:47.584408+0000","last_clean":"2024-05-03T18:45:47.584408+0000","last_became_active":"2024-05-03T18:44:49.438573+0000","last_became_peered":"2024-05-03T18:44:49.438573+0000","last_unstale":"2024-05-03T18:45:47.584408+0000","last_undegraded":"2024-05-03T18:45:47.584408+0000","last_fullsized":"2024-05-03T18:45:47.584408+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T06:09:31.042649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"78'2","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.585857+0000","last_change":"2024-05-03T18:44:46.406752+0000","last_active":"2024-05-03T18:45:47.585857+0000","last_peered":"2024-05-03T18:45:47.585857+0000","last_clean":"2024-05-03T18:45:47.585857+0000","last_became_active":"2024-05-03T18:44:46.406288+0000","last_became_peered":"2024-05-03T18:44:46.406288+0000","last_unstale":"2024-05-03T18:45:47.585857+0000","last_undegraded":"2024-05-03T18:45:47.585857+0000","last_fullsized":"2024-05-03T18:45:47.585857+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_clean_scrub_stamp":"2024-05-03T18:44:45.380544+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:20:33.947006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.588049+0000","last_change":"2024-05-03T18:45:47.588049+0000","last_active":"2024-05-03T18:45:47.588049+0000","last_peered":"2024-05-03T18:45:47.588049+0000","last_clean":"2024-05-03T18:45:47.588049+0000","last_became_active":"2024-05-03T18:45:47.587534+0000","last_became_peered":"2024-05-03T18:45:47.587534+0000","last_unstale":"2024-05-03T18:45:47.588049+0000","last_undegraded":"2024-05-03T18:45:47.588049+0000","last_fullsized":"2024-05-03T18:45:47.588049+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:16:36.601533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7],"acting":[11,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.584305+0000","last_change":"2024-05-03T18:44:46.647155+0000","last_active":"2024-05-03T18:45:47.584305+0000","last_peered":"2024-05-03T18:45:47.584305+0000","last_clean":"2024-05-03T18:45:47.584305+0000","last_became_active":"2024-05-03T18:44:46.646719+0000","last_became_peered":"2024-05-03T18:44:46.646719+0000","last_unstale":"2024-05-03T18:45:47.584305+0000","last_undegraded":"2024-05-03T18:45:47.584305+0000","last_fullsized":"2024-05-03T18:45:47.584305+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_clean_scrub_stamp":"2024-05-03T18:44:45.380544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T18:49:13.701498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"78'1","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.586646+0000","last_change":"2024-05-03T18:44:49.423776+0000","last_active":"2024-05-03T18:45:47.586646+0000","last_peered":"2024-05-03T18:45:47.586646+0000","last_clean":"2024-05-03T18:45:47.586646+0000","last_became_active":"2024-05-03T18:44:49.423414+0000","last_became_peered":"2024-05-03T18:44:49.423414+0000","last_unstale":"2024-05-03T18:45:47.586646+0000","last_undegraded":"2024-05-03T18:45:47.586646+0000","last_fullsized":"2024-05-03T18:45:47.586646+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:48:03.556270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"78'1","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.584512+0000","last_change":"2024-05-03T18:44:46.407725+0000","last_active":"2024-05-03T18:45:47.584512+0000","last_peered":"2024-05-03T18:45:47.584512+0000","last_clean":"2024-05-03T18:45:47.584512+0000","last_became_active":"2024-05-03T18:44:46.407317+0000","last_became_peered":"2024-05-03T18:44:46.407317+0000","last_unstale":"2024-05-03T18:45:47.584512+0000","last_undegraded":"2024-05-03T18:45:47.584512+0000","last_fullsized":"2024-05-03T18:45:47.584512+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_clean_scrub_stamp":"2024-05-03T18:44:45.380544+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:02:32.739918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.586698+0000","last_change":"2024-05-03T18:44:49.432439+0000","last_active":"2024-05-03T18:45:47.586698+0000","last_peered":"2024-05-03T18:45:47.586698+0000","last_clean":"2024-05-03T18:45:47.586698+0000","last_became_active":"2024-05-03T18:44:49.431792+0000","last_became_peered":"2024-05-03T18:44:49.431792+0000","last_unstale":"2024-05-03T18:45:47.586698+0000","last_undegraded":"2024-05-03T18:45:47.586698+0000","last_fullsized":"2024-05-03T18:45:47.586698+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:12:46.223514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.584147+0000","last_change":"2024-05-03T18:44:49.440075+0000","last_active":"2024-05-03T18:45:47.584147+0000","last_peered":"2024-05-03T18:45:47.584147+0000","last_clean":"2024-05-03T18:45:47.584147+0000","last_became_active":"2024-05-03T18:44:49.439655+0000","last_became_peered":"2024-05-03T18:44:49.439655+0000","last_unstale":"2024-05-03T18:45:47.584147+0000","last_undegraded":"2024-05-03T18:45:47.584147+0000","last_fullsized":"2024-05-03T18:45:47.584147+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:19:03.306588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"78'4","reported_seq":120,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:46:33.614206+0000","last_change":"2024-05-03T18:44:46.645090+0000","last_active":"2024-05-03T18:46:33.614206+0000","last_peered":"2024-05-03T18:46:33.614206+0000","last_clean":"2024-05-03T18:46:33.614206+0000","last_became_active":"2024-05-03T18:44:46.644722+0000","last_became_peered":"2024-05-03T18:44:46.644722+0000","last_unstale":"2024-05-03T18:46:33.614206+0000","last_undegraded":"2024-05-03T18:46:33.614206+0000","last_fullsized":"2024-05-03T18:46:33.614206+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_clean_scrub_stamp":"2024-05-03T18:44:45.380544+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:44:38.032522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":90,"num_read_kb":90,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.586922+0000","last_change":"2024-05-03T18:45:47.586554+0000","last_active":"2024-05-03T18:45:47.586922+0000","last_peered":"2024-05-03T18:45:47.586922+0000","last_clean":"2024-05-03T18:45:47.586922+0000","last_became_active":"2024-05-03T18:45:47.586247+0000","last_became_peered":"2024-05-03T18:45:47.586247+0000","last_unstale":"2024-05-03T18:45:47.586922+0000","last_undegraded":"2024-05-03T18:45:47.586922+0000","last_fullsized":"2024-05-03T18:45:47.586922+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T06:30:16.743598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.3","version":"78'3","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.635251+0000","last_change":"2024-05-03T18:44:46.402866+0000","last_active":"2024-05-03T18:45:47.635251+0000","last_peered":"2024-05-03T18:45:47.635251+0000","last_clean":"2024-05-03T18:45:47.635251+0000","last_became_active":"2024-05-03T18:44:46.402394+0000","last_became_peered":"2024-05-03T18:44:46.402394+0000","last_unstale":"2024-05-03T18:45:47.635251+0000","last_undegraded":"2024-05-03T18:45:47.635251+0000","last_fullsized":"2024-05-03T18:45:47.635251+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_clean_scrub_stamp":"2024-05-03T18:44:45.380544+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:28:37.257537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":101,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.584399+0000","last_change":"2024-05-03T18:43:28.584846+0000","last_active":"2024-05-03T18:45:47.584399+0000","last_peered":"2024-05-03T18:45:47.584399+0000","last_clean":"2024-05-03T18:45:47.584399+0000","last_became_active":"2024-05-03T18:43:28.580087+0000","last_became_peered":"2024-05-03T18:43:28.580087+0000","last_unstale":"2024-05-03T18:45:47.584399+0000","last_undegraded":"2024-05-03T18:45:47.584399+0000","last_fullsized":"2024-05-03T18:45:47.584399+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:40:08.599278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:40:08.599278+0000","last_clean_scrub_stamp":"2024-05-03T18:40:08.599278+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:14:31.836127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.956184+0000","last_change":"2024-05-03T18:44:49.438545+0000","last_active":"2024-05-03T18:45:47.956184+0000","last_peered":"2024-05-03T18:45:47.956184+0000","last_clean":"2024-05-03T18:45:47.956184+0000","last_became_active":"2024-05-03T18:44:49.438178+0000","last_became_peered":"2024-05-03T18:44:49.438178+0000","last_unstale":"2024-05-03T18:45:47.956184+0000","last_undegraded":"2024-05-03T18:45:47.956184+0000","last_fullsized":"2024-05-03T18:45:47.956184+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:55:10.320262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"78'3","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.584446+0000","last_change":"2024-05-03T18:44:46.409567+0000","last_active":"2024-05-03T18:45:47.584446+0000","last_peered":"2024-05-03T18:45:47.584446+0000","last_clean":"2024-05-03T18:45:47.584446+0000","last_became_active":"2024-05-03T18:44:46.409078+0000","last_became_peered":"2024-05-03T18:44:46.409078+0000","last_unstale":"2024-05-03T18:45:47.584446+0000","last_undegraded":"2024-05-03T18:45:47.584446+0000","last_fullsized":"2024-05-03T18:45:47.584446+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_clean_scrub_stamp":"2024-05-03T18:44:45.380544+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:51:45.865890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"78'1","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.585437+0000","last_change":"2024-05-03T18:44:46.645061+0000","last_active":"2024-05-03T18:45:47.585437+0000","last_peered":"2024-05-03T18:45:47.585437+0000","last_clean":"2024-05-03T18:45:47.585437+0000","last_became_active":"2024-05-03T18:44:46.644631+0000","last_became_peered":"2024-05-03T18:44:46.644631+0000","last_unstale":"2024-05-03T18:45:47.585437+0000","last_undegraded":"2024-05-03T18:45:47.585437+0000","last_fullsized":"2024-05-03T18:45:47.585437+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_clean_scrub_stamp":"2024-05-03T18:44:45.380544+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:19:37.285434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.956494+0000","last_change":"2024-05-03T18:44:49.435674+0000","last_active":"2024-05-03T18:45:47.956494+0000","last_peered":"2024-05-03T18:45:47.956494+0000","last_clean":"2024-05-03T18:45:47.956494+0000","last_became_active":"2024-05-03T18:44:49.435333+0000","last_became_peered":"2024-05-03T18:44:49.435333+0000","last_unstale":"2024-05-03T18:45:47.956494+0000","last_undegraded":"2024-05-03T18:45:47.956494+0000","last_fullsized":"2024-05-03T18:45:47.956494+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:05:20.283773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"78'10","reported_seq":35,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.584714+0000","last_change":"2024-05-03T18:44:46.402696+0000","last_active":"2024-05-03T18:45:47.584714+0000","last_peered":"2024-05-03T18:45:47.584714+0000","last_clean":"2024-05-03T18:45:47.584714+0000","last_became_active":"2024-05-03T18:44:46.402288+0000","last_became_peered":"2024-05-03T18:44:46.402288+0000","last_unstale":"2024-05-03T18:45:47.584714+0000","last_undegraded":"2024-05-03T18:45:47.584714+0000","last_fullsized":"2024-05-03T18:45:47.584714+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_clean_scrub_stamp":"2024-05-03T18:44:45.380544+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:45:32.200586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.582839+0000","last_change":"2024-05-03T18:44:49.422754+0000","last_active":"2024-05-03T18:45:47.582839+0000","last_peered":"2024-05-03T18:45:47.582839+0000","last_clean":"2024-05-03T18:45:47.582839+0000","last_became_active":"2024-05-03T18:44:49.422385+0000","last_became_peered":"2024-05-03T18:44:49.422385+0000","last_unstale":"2024-05-03T18:45:47.582839+0000","last_undegraded":"2024-05-03T18:45:47.582839+0000","last_fullsized":"2024-05-03T18:45:47.582839+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:07:35.065842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"78'4","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.635326+0000","last_change":"2024-05-03T18:44:47.135851+0000","last_active":"2024-05-03T18:45:47.635326+0000","last_peered":"2024-05-03T18:45:47.635326+0000","last_clean":"2024-05-03T18:45:47.635326+0000","last_became_active":"2024-05-03T18:44:47.135480+0000","last_became_peered":"2024-05-03T18:44:47.135480+0000","last_unstale":"2024-05-03T18:45:47.635326+0000","last_undegraded":"2024-05-03T18:45:47.635326+0000","last_fullsized":"2024-05-03T18:45:47.635326+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_clean_scrub_stamp":"2024-05-03T18:44:45.380544+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:50:55.582018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.586246+0000","last_change":"2024-05-03T18:44:49.433123+0000","last_active":"2024-05-03T18:45:47.586246+0000","last_peered":"2024-05-03T18:45:47.586246+0000","last_clean":"2024-05-03T18:45:47.586246+0000","last_became_active":"2024-05-03T18:44:49.432781+0000","last_became_peered":"2024-05-03T18:44:49.432781+0000","last_unstale":"2024-05-03T18:45:47.586246+0000","last_undegraded":"2024-05-03T18:45:47.586246+0000","last_fullsized":"2024-05-03T18:45:47.586246+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:30:49.781052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"78'2","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.582904+0000","last_change":"2024-05-03T18:44:46.645267+0000","last_active":"2024-05-03T18:45:47.582904+0000","last_peered":"2024-05-03T18:45:47.582904+0000","last_clean":"2024-05-03T18:45:47.582904+0000","last_became_active":"2024-05-03T18:44:46.644804+0000","last_became_peered":"2024-05-03T18:44:46.644804+0000","last_unstale":"2024-05-03T18:45:47.582904+0000","last_undegraded":"2024-05-03T18:45:47.582904+0000","last_fullsized":"2024-05-03T18:45:47.582904+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_clean_scrub_stamp":"2024-05-03T18:44:45.380544+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:10:59.262218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.585467+0000","last_change":"2024-05-03T18:44:49.439352+0000","last_active":"2024-05-03T18:45:47.585467+0000","last_peered":"2024-05-03T18:45:47.585467+0000","last_clean":"2024-05-03T18:45:47.585467+0000","last_became_active":"2024-05-03T18:44:49.438981+0000","last_became_peered":"2024-05-03T18:44:49.438981+0000","last_unstale":"2024-05-03T18:45:47.585467+0000","last_undegraded":"2024-05-03T18:45:47.585467+0000","last_fullsized":"2024-05-03T18:45:47.585467+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T06:04:25.566153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"78'3","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.582432+0000","last_change":"2024-05-03T18:44:46.402305+0000","last_active":"2024-05-03T18:45:47.582432+0000","last_peered":"2024-05-03T18:45:47.582432+0000","last_clean":"2024-05-03T18:45:47.582432+0000","last_became_active":"2024-05-03T18:44:46.401633+0000","last_became_peered":"2024-05-03T18:44:46.401633+0000","last_unstale":"2024-05-03T18:45:47.582432+0000","last_undegraded":"2024-05-03T18:45:47.582432+0000","last_fullsized":"2024-05-03T18:45:47.582432+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_clean_scrub_stamp":"2024-05-03T18:44:45.380544+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:55:06.025508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.584759+0000","last_change":"2024-05-03T18:44:49.436728+0000","last_active":"2024-05-03T18:45:47.584759+0000","last_peered":"2024-05-03T18:45:47.584759+0000","last_clean":"2024-05-03T18:45:47.584759+0000","last_became_active":"2024-05-03T18:44:49.436356+0000","last_became_peered":"2024-05-03T18:44:49.436356+0000","last_unstale":"2024-05-03T18:45:47.584759+0000","last_undegraded":"2024-05-03T18:45:47.584759+0000","last_fullsized":"2024-05-03T18:45:47.584759+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:31:34.827237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"78'1","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.588472+0000","last_change":"2024-05-03T18:44:46.404949+0000","last_active":"2024-05-03T18:45:47.588472+0000","last_peered":"2024-05-03T18:45:47.588472+0000","last_clean":"2024-05-03T18:45:47.588472+0000","last_became_active":"2024-05-03T18:44:46.404516+0000","last_became_peered":"2024-05-03T18:44:46.404516+0000","last_unstale":"2024-05-03T18:45:47.588472+0000","last_undegraded":"2024-05-03T18:45:47.588472+0000","last_fullsized":"2024-05-03T18:45:47.588472+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_clean_scrub_stamp":"2024-05-03T18:44:45.380544+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:02:16.070763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.582603+0000","last_change":"2024-05-03T18:44:49.424090+0000","last_active":"2024-05-03T18:45:47.582603+0000","last_peered":"2024-05-03T18:45:47.582603+0000","last_clean":"2024-05-03T18:45:47.582603+0000","last_became_active":"2024-05-03T18:44:49.423110+0000","last_became_peered":"2024-05-03T18:44:49.423110+0000","last_unstale":"2024-05-03T18:45:47.582603+0000","last_undegraded":"2024-05-03T18:45:47.582603+0000","last_fullsized":"2024-05-03T18:45:47.582603+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:27:19.600924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"78'4","reported_seq":42,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.639194+0000","last_change":"2024-05-03T18:45:47.639194+0000","last_active":"2024-05-03T18:45:47.639194+0000","last_peered":"2024-05-03T18:45:47.639194+0000","last_clean":"2024-05-03T18:45:47.639194+0000","last_became_active":"2024-05-03T18:45:47.636067+0000","last_became_peered":"2024-05-03T18:45:47.636067+0000","last_unstale":"2024-05-03T18:45:47.639194+0000","last_undegraded":"2024-05-03T18:45:47.639194+0000","last_fullsized":"2024-05-03T18:45:47.639194+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_clean_scrub_stamp":"2024-05-03T18:44:45.380544+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:58:31.576713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.584592+0000","last_change":"2024-05-03T18:44:49.432498+0000","last_active":"2024-05-03T18:45:47.584592+0000","last_peered":"2024-05-03T18:45:47.584592+0000","last_clean":"2024-05-03T18:45:47.584592+0000","last_became_active":"2024-05-03T18:44:49.432152+0000","last_became_peered":"2024-05-03T18:44:49.432152+0000","last_unstale":"2024-05-03T18:45:47.584592+0000","last_undegraded":"2024-05-03T18:45:47.584592+0000","last_fullsized":"2024-05-03T18:45:47.584592+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:07:35.058213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"78'3","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.588551+0000","last_change":"2024-05-03T18:44:47.135973+0000","last_active":"2024-05-03T18:45:47.588551+0000","last_peered":"2024-05-03T18:45:47.588551+0000","last_clean":"2024-05-03T18:45:47.588551+0000","last_became_active":"2024-05-03T18:44:47.135548+0000","last_became_peered":"2024-05-03T18:44:47.135548+0000","last_unstale":"2024-05-03T18:45:47.588551+0000","last_undegraded":"2024-05-03T18:45:47.588551+0000","last_fullsized":"2024-05-03T18:45:47.588551+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_clean_scrub_stamp":"2024-05-03T18:44:45.380544+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:06:42.858462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.585036+0000","last_change":"2024-05-03T18:44:49.432722+0000","last_active":"2024-05-03T18:45:47.585036+0000","last_peered":"2024-05-03T18:45:47.585036+0000","last_clean":"2024-05-03T18:45:47.585036+0000","last_became_active":"2024-05-03T18:44:49.432393+0000","last_became_peered":"2024-05-03T18:44:49.432393+0000","last_unstale":"2024-05-03T18:45:47.585036+0000","last_undegraded":"2024-05-03T18:45:47.585036+0000","last_fullsized":"2024-05-03T18:45:47.585036+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:44:20.551442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"78'1","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.584489+0000","last_change":"2024-05-03T18:44:46.402489+0000","last_active":"2024-05-03T18:45:47.584489+0000","last_peered":"2024-05-03T18:45:47.584489+0000","last_clean":"2024-05-03T18:45:47.584489+0000","last_became_active":"2024-05-03T18:44:46.401992+0000","last_became_peered":"2024-05-03T18:44:46.401992+0000","last_unstale":"2024-05-03T18:45:47.584489+0000","last_undegraded":"2024-05-03T18:45:47.584489+0000","last_fullsized":"2024-05-03T18:45:47.584489+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_clean_scrub_stamp":"2024-05-03T18:44:45.380544+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:09:10.600254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"78'1","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.592504+0000","last_change":"2024-05-03T18:45:47.592504+0000","last_active":"2024-05-03T18:45:47.592504+0000","last_peered":"2024-05-03T18:45:47.592504+0000","last_clean":"2024-05-03T18:45:47.592504+0000","last_became_active":"2024-05-03T18:45:47.589068+0000","last_became_peered":"2024-05-03T18:45:47.589068+0000","last_unstale":"2024-05-03T18:45:47.592504+0000","last_undegraded":"2024-05-03T18:45:47.592504+0000","last_fullsized":"2024-05-03T18:45:47.592504+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:52:18.605331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,8],"acting":[7,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"78'2","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.956448+0000","last_change":"2024-05-03T18:44:46.404414+0000","last_active":"2024-05-03T18:45:47.956448+0000","last_peered":"2024-05-03T18:45:47.956448+0000","last_clean":"2024-05-03T18:45:47.956448+0000","last_became_active":"2024-05-03T18:44:46.404136+0000","last_became_peered":"2024-05-03T18:44:46.404136+0000","last_unstale":"2024-05-03T18:45:47.956448+0000","last_undegraded":"2024-05-03T18:45:47.956448+0000","last_fullsized":"2024-05-03T18:45:47.956448+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_clean_scrub_stamp":"2024-05-03T18:44:45.380544+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:24:58.125984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.584925+0000","last_change":"2024-05-03T18:44:49.429013+0000","last_active":"2024-05-03T18:45:47.584925+0000","last_peered":"2024-05-03T18:45:47.584925+0000","last_clean":"2024-05-03T18:45:47.584925+0000","last_became_active":"2024-05-03T18:44:49.428380+0000","last_became_peered":"2024-05-03T18:44:49.428380+0000","last_unstale":"2024-05-03T18:45:47.584925+0000","last_undegraded":"2024-05-03T18:45:47.584925+0000","last_fullsized":"2024-05-03T18:45:47.584925+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:13:00.073384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.584869+0000","last_change":"2024-05-03T18:44:49.437186+0000","last_active":"2024-05-03T18:45:47.584869+0000","last_peered":"2024-05-03T18:45:47.584869+0000","last_clean":"2024-05-03T18:45:47.584869+0000","last_became_active":"2024-05-03T18:44:49.435192+0000","last_became_peered":"2024-05-03T18:44:49.435192+0000","last_unstale":"2024-05-03T18:45:47.584869+0000","last_undegraded":"2024-05-03T18:45:47.584869+0000","last_fullsized":"2024-05-03T18:45:47.584869+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:16:25.707746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.589270+0000","last_change":"2024-05-03T18:44:49.441553+0000","last_active":"2024-05-03T18:45:47.589270+0000","last_peered":"2024-05-03T18:45:47.589270+0000","last_clean":"2024-05-03T18:45:47.589270+0000","last_became_active":"2024-05-03T18:44:49.441155+0000","last_became_peered":"2024-05-03T18:44:49.441155+0000","last_unstale":"2024-05-03T18:45:47.589270+0000","last_undegraded":"2024-05-03T18:45:47.589270+0000","last_fullsized":"2024-05-03T18:45:47.589270+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:53:53.888748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.588350+0000","last_change":"2024-05-03T18:44:49.436641+0000","last_active":"2024-05-03T18:45:47.588350+0000","last_peered":"2024-05-03T18:45:47.588350+0000","last_clean":"2024-05-03T18:45:47.588350+0000","last_became_active":"2024-05-03T18:44:49.436232+0000","last_became_peered":"2024-05-03T18:44:49.436232+0000","last_unstale":"2024-05-03T18:45:47.588350+0000","last_undegraded":"2024-05-03T18:45:47.588350+0000","last_fullsized":"2024-05-03T18:45:47.588350+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:06:55.773088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.586032+0000","last_change":"2024-05-03T18:44:49.434394+0000","last_active":"2024-05-03T18:45:47.586032+0000","last_peered":"2024-05-03T18:45:47.586032+0000","last_clean":"2024-05-03T18:45:47.586032+0000","last_became_active":"2024-05-03T18:44:49.433995+0000","last_became_peered":"2024-05-03T18:44:49.433995+0000","last_unstale":"2024-05-03T18:45:47.586032+0000","last_undegraded":"2024-05-03T18:45:47.586032+0000","last_fullsized":"2024-05-03T18:45:47.586032+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:25:20.478426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.956595+0000","last_change":"2024-05-03T18:44:49.439218+0000","last_active":"2024-05-03T18:45:47.956595+0000","last_peered":"2024-05-03T18:45:47.956595+0000","last_clean":"2024-05-03T18:45:47.956595+0000","last_became_active":"2024-05-03T18:44:49.438875+0000","last_became_peered":"2024-05-03T18:44:49.438875+0000","last_unstale":"2024-05-03T18:45:47.956595+0000","last_undegraded":"2024-05-03T18:45:47.956595+0000","last_fullsized":"2024-05-03T18:45:47.956595+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:08:55.899829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.584927+0000","last_change":"2024-05-03T18:44:49.433079+0000","last_active":"2024-05-03T18:45:47.584927+0000","last_peered":"2024-05-03T18:45:47.584927+0000","last_clean":"2024-05-03T18:45:47.584927+0000","last_became_active":"2024-05-03T18:44:49.432740+0000","last_became_peered":"2024-05-03T18:44:49.432740+0000","last_unstale":"2024-05-03T18:45:47.584927+0000","last_undegraded":"2024-05-03T18:45:47.584927+0000","last_fullsized":"2024-05-03T18:45:47.584927+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:31:36.317667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.589311+0000","last_change":"2024-05-03T18:44:49.435321+0000","last_active":"2024-05-03T18:45:47.589311+0000","last_peered":"2024-05-03T18:45:47.589311+0000","last_clean":"2024-05-03T18:45:47.589311+0000","last_became_active":"2024-05-03T18:44:49.434935+0000","last_became_peered":"2024-05-03T18:44:49.434935+0000","last_unstale":"2024-05-03T18:45:47.589311+0000","last_undegraded":"2024-05-03T18:45:47.589311+0000","last_fullsized":"2024-05-03T18:45:47.589311+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:14:32.406164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.582800+0000","last_change":"2024-05-03T18:44:49.438202+0000","last_active":"2024-05-03T18:45:47.582800+0000","last_peered":"2024-05-03T18:45:47.582800+0000","last_clean":"2024-05-03T18:45:47.582800+0000","last_became_active":"2024-05-03T18:44:49.437822+0000","last_became_peered":"2024-05-03T18:44:49.437822+0000","last_unstale":"2024-05-03T18:45:47.582800+0000","last_undegraded":"2024-05-03T18:45:47.582800+0000","last_fullsized":"2024-05-03T18:45:47.582800+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:12:54.417022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.583991+0000","last_change":"2024-05-03T18:44:49.434992+0000","last_active":"2024-05-03T18:45:47.583991+0000","last_peered":"2024-05-03T18:45:47.583991+0000","last_clean":"2024-05-03T18:45:47.583991+0000","last_became_active":"2024-05-03T18:44:49.434668+0000","last_became_peered":"2024-05-03T18:44:49.434668+0000","last_unstale":"2024-05-03T18:45:47.583991+0000","last_undegraded":"2024-05-03T18:45:47.583991+0000","last_fullsized":"2024-05-03T18:45:47.583991+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:20:48.527470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.582372+0000","last_change":"2024-05-03T18:44:49.432690+0000","last_active":"2024-05-03T18:45:47.582372+0000","last_peered":"2024-05-03T18:45:47.582372+0000","last_clean":"2024-05-03T18:45:47.582372+0000","last_became_active":"2024-05-03T18:44:49.432389+0000","last_became_peered":"2024-05-03T18:44:49.432389+0000","last_unstale":"2024-05-03T18:45:47.582372+0000","last_undegraded":"2024-05-03T18:45:47.582372+0000","last_fullsized":"2024-05-03T18:45:47.582372+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:51:27.326901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.583214+0000","last_change":"2024-05-03T18:44:49.434205+0000","last_active":"2024-05-03T18:45:47.583214+0000","last_peered":"2024-05-03T18:45:47.583214+0000","last_clean":"2024-05-03T18:45:47.583214+0000","last_became_active":"2024-05-03T18:44:49.433619+0000","last_became_peered":"2024-05-03T18:44:49.433619+0000","last_unstale":"2024-05-03T18:45:47.583214+0000","last_undegraded":"2024-05-03T18:45:47.583214+0000","last_fullsized":"2024-05-03T18:45:47.583214+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:36:15.933276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.956486+0000","last_change":"2024-05-03T18:44:49.425276+0000","last_active":"2024-05-03T18:45:47.956486+0000","last_peered":"2024-05-03T18:45:47.956486+0000","last_clean":"2024-05-03T18:45:47.956486+0000","last_became_active":"2024-05-03T18:44:49.424916+0000","last_became_peered":"2024-05-03T18:44:49.424916+0000","last_unstale":"2024-05-03T18:45:47.956486+0000","last_undegraded":"2024-05-03T18:45:47.956486+0000","last_fullsized":"2024-05-03T18:45:47.956486+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:09:03.512860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.585105+0000","last_change":"2024-05-03T18:44:49.433335+0000","last_active":"2024-05-03T18:45:47.585105+0000","last_peered":"2024-05-03T18:45:47.585105+0000","last_clean":"2024-05-03T18:45:47.585105+0000","last_became_active":"2024-05-03T18:44:49.432991+0000","last_became_peered":"2024-05-03T18:44:49.432991+0000","last_unstale":"2024-05-03T18:45:47.585105+0000","last_undegraded":"2024-05-03T18:45:47.585105+0000","last_fullsized":"2024-05-03T18:45:47.585105+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:31:25.182590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":55834,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":577,"num_read_kb":347,"num_write":159,"num_write_kb":101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":167502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":136,"ondisk_log_size":136,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710756,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27876,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717276,"statfs":{"total":95995035648,"available":95966490624,"internally_reserved":0,"allocated":872448,"data_stored":425312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906986,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28548,"kb_used_data":1524,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716604,"statfs":{"total":95995035648,"available":95965802496,"internally_reserved":0,"allocated":1560576,"data_stored":1068626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070510,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":888832,"data_stored":425548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266739,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27892,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717260,"statfs":{"total":95995035648,"available":95966474240,"internally_reserved":0,"allocated":892928,"data_stored":425585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462969,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40952,"kb_used_data":1448,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704200,"statfs":{"total":95995035648,"available":95953100800,"internally_reserved":0,"allocated":1482752,"data_stored":1017398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626492,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27880,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717272,"statfs":{"total":95995035648,"available":95966486528,"internally_reserved":0,"allocated":876544,"data_stored":424937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574934,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":929792,"data_stored":476563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738458,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36216,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708936,"statfs":{"total":95995035648,"available":95957950464,"internally_reserved":0,"allocated":897024,"data_stored":426208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411409,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":954368,"data_stored":478304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182476,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36208,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708944,"statfs":{"total":95995035648,"available":95957958656,"internally_reserved":0,"allocated":892928,"data_stored":425248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986248,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":880640,"data_stored":425374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822722,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36796,"kb_used_data":1456,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708356,"statfs":{"total":95995035648,"available":95957356544,"internally_reserved":0,"allocated":1490944,"data_stored":1016965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":51833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":53574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":53411,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-03T18:46:40.563 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid f500455e-097b-11ef-bc97-c7b262605968 -- ceph pg dump --format=json 2024-05-03T18:46:40.816 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:46:41.204 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:46:41 smithi032 ceph-mon[29763]: from='client.14928 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-03T18:46:41.204 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:46:41 smithi032 ceph-mon[29763]: pgmap v337: 97 pgs: 97 active+clean; 631 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:46:41.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:46:41 smithi045 ceph-mon[33299]: from='client.14928 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-03T18:46:41.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:46:41 smithi045 ceph-mon[33299]: pgmap v337: 97 pgs: 97 active+clean; 631 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:46:41.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:46:41 smithi113 ceph-mon[33540]: from='client.14928 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-03T18:46:41.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:46:41 smithi113 ceph-mon[33540]: pgmap v337: 97 pgs: 97 active+clean; 631 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:46:42.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:42.356+0000 7f2d11c28700 1 -- 172.21.15.32:0/3916259856 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d0c107420 msgr2=0x7f2d0c107800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:42.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:42.356+0000 7f2d11c28700 1 --2- 172.21.15.32:0/3916259856 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d0c107420 0x7f2d0c107800 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f2d00009990 tx=0x7f2d00009ca0 comp rx=0 tx=0).stop 2024-05-03T18:46:42.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:42.357+0000 7f2d11c28700 1 -- 172.21.15.32:0/3916259856 shutdown_connections 2024-05-03T18:46:42.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:42.357+0000 7f2d11c28700 1 --2- 172.21.15.32:0/3916259856 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2d0c1030b0 0x7f2d0c10f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:42.358 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:42.357+0000 7f2d11c28700 1 --2- 172.21.15.32:0/3916259856 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d0c107420 0x7f2d0c107800 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:42.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:42.357+0000 7f2d11c28700 1 --2- 172.21.15.32:0/3916259856 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2d0c1026f0 0x7f2d0c102b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:42.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:42.357+0000 7f2d11c28700 1 -- 172.21.15.32:0/3916259856 >> 172.21.15.32:0/3916259856 conn(0x7f2d0c078210 msgr2=0x7f2d0c078620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:46:42.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:42.357+0000 7f2d11c28700 1 -- 172.21.15.32:0/3916259856 shutdown_connections 2024-05-03T18:46:42.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:42.357+0000 7f2d11c28700 1 -- 172.21.15.32:0/3916259856 wait complete. 2024-05-03T18:46:42.359 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:42.357+0000 7f2d11c28700 1 Processor -- start 2024-05-03T18:46:42.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:42.358+0000 7f2d11c28700 1 -- start start 2024-05-03T18:46:42.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:42.358+0000 7f2d11c28700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2d0c1026f0 0x7f2d0c19e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:42.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:42.358+0000 7f2d11c28700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d0c1030b0 0x7f2d0c19ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:42.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:42.358+0000 7f2d11c28700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2d0c107420 0x7f2d0c1989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:42.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:42.358+0000 7f2d11c28700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f2d0c1126a0 con 0x7f2d0c1030b0 2024-05-03T18:46:42.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:42.358+0000 7f2d11c28700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2d0c112520 con 0x7f2d0c107420 2024-05-03T18:46:42.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:42.358+0000 7f2d11c28700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f2d0c112820 con 0x7f2d0c1026f0 2024-05-03T18:46:42.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:42.358+0000 7f2d0affd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d0c1030b0 0x7f2d0c19ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:42.361 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:42.358+0000 7f2d0affd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d0c1030b0 0x7f2d0c19ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46564/0 (socket says 172.21.15.32:46564) 2024-05-03T18:46:42.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:42.358+0000 7f2d0affd700 1 -- 172.21.15.32:0/1677493507 learned_addr learned my addr 172.21.15.32:0/1677493507 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:46:42.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:42.358+0000 7f2d0b7fe700 1 --2- 172.21.15.32:0/1677493507 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2d0c1026f0 0x7f2d0c19e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:42.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:42.358+0000 7f2d0bfff700 1 --2- 172.21.15.32:0/1677493507 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2d0c107420 0x7f2d0c1989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:42.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:42.358+0000 7f2d0affd700 1 -- 172.21.15.32:0/1677493507 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2d0c1026f0 msgr2=0x7f2d0c19e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:42.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:42.358+0000 7f2d0affd700 1 --2- 172.21.15.32:0/1677493507 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2d0c1026f0 0x7f2d0c19e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:42.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:42.358+0000 7f2d0affd700 1 -- 172.21.15.32:0/1677493507 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2d0c107420 msgr2=0x7f2d0c1989e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:42.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:42.358+0000 7f2d0affd700 1 --2- 172.21.15.32:0/1677493507 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2d0c107420 0x7f2d0c1989e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:42.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:42.358+0000 7f2d0affd700 1 -- 172.21.15.32:0/1677493507 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d0c199140 con 0x7f2d0c1030b0 2024-05-03T18:46:42.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:42.358+0000 7f2d0b7fe700 1 --2- 172.21.15.32:0/1677493507 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2d0c1026f0 0x7f2d0c19e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T18:46:42.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:42.359+0000 7f2d0affd700 1 --2- 172.21.15.32:0/1677493507 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d0c1030b0 0x7f2d0c19ed60 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc00b530 tx=0x7f2cfc00b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:46:42.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:42.359+0000 7f2d08ff9700 1 -- 172.21.15.32:0/1677493507 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cfc015020 con 0x7f2d0c1030b0 2024-05-03T18:46:42.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:42.359+0000 7f2d08ff9700 1 -- 172.21.15.32:0/1677493507 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2cfc011ce0 con 0x7f2d0c1030b0 2024-05-03T18:46:42.362 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:42.359+0000 7f2d08ff9700 1 -- 172.21.15.32:0/1677493507 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cfc010d20 con 0x7f2d0c1030b0 2024-05-03T18:46:42.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:42.359+0000 7f2d11c28700 1 -- 172.21.15.32:0/1677493507 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d0c199430 con 0x7f2d0c1030b0 2024-05-03T18:46:42.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:42.359+0000 7f2d11c28700 1 -- 172.21.15.32:0/1677493507 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d0c06bc50 con 0x7f2d0c1030b0 2024-05-03T18:46:42.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:42.360+0000 7f2d11c28700 1 -- 172.21.15.32:0/1677493507 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d0c103b10 con 0x7f2d0c1030b0 2024-05-03T18:46:42.363 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:42.361+0000 7f2d08ff9700 1 -- 172.21.15.32:0/1677493507 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f2cfc018020 con 0x7f2d0c1030b0 2024-05-03T18:46:42.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:42.362+0000 7f2d08ff9700 1 --2- 172.21.15.32:0/1677493507 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f2cf40b2990 0x7f2cf40b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:42.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:42.362+0000 7f2d0b7fe700 1 --2- 172.21.15.32:0/1677493507 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f2cf40b2990 0x7f2cf40b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:42.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:42.362+0000 7f2d08ff9700 1 -- 172.21.15.32:0/1677493507 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8269+0+0 (secure 0 0 0) 0x7f2cfc0e1fe0 con 0x7f2d0c1030b0 2024-05-03T18:46:42.364 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:42.363+0000 7f2d0b7fe700 1 --2- 172.21.15.32:0/1677493507 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f2cf40b2990 0x7f2cf40b4e50 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f2d00006010 tx=0x7f2d0001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:46:42.366 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:42.366+0000 7f2d08ff9700 1 -- 172.21.15.32:0/1677493507 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f2cfc01f020 con 0x7f2d0c1030b0 2024-05-03T18:46:42.577 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:42.576+0000 7f2d11c28700 1 -- 172.21.15.32:0/1677493507 --> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2d0c06c720 con 0x7f2cf40b2990 2024-05-03T18:46:42.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:42.583+0000 7f2d08ff9700 1 -- 172.21.15.32:0/1677493507 <== mgr.14152 v2:172.21.15.32:6800/1796547904 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258703 (secure 0 0 0) 0x7f2d0c06c720 con 0x7f2cf40b2990 2024-05-03T18:46:42.587 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-03T18:46:42.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:42.586+0000 7f2d11c28700 1 -- 172.21.15.32:0/1677493507 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f2cf40b2990 msgr2=0x7f2cf40b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:42.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:42.586+0000 7f2d11c28700 1 --2- 172.21.15.32:0/1677493507 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f2cf40b2990 0x7f2cf40b4e50 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f2d00006010 tx=0x7f2d0001a040 comp rx=0 tx=0).stop 2024-05-03T18:46:42.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:42.586+0000 7f2d11c28700 1 -- 172.21.15.32:0/1677493507 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d0c1030b0 msgr2=0x7f2d0c19ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:42.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:42.586+0000 7f2d11c28700 1 --2- 172.21.15.32:0/1677493507 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d0c1030b0 0x7f2d0c19ed60 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc00b530 tx=0x7f2cfc00b8f0 comp rx=0 tx=0).stop 2024-05-03T18:46:42.589 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:42.586+0000 7f2d11c28700 1 -- 172.21.15.32:0/1677493507 shutdown_connections 2024-05-03T18:46:42.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:42.586+0000 7f2d11c28700 1 --2- 172.21.15.32:0/1677493507 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f2cf40b2990 0x7f2cf40b4e50 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:42.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:42.586+0000 7f2d11c28700 1 --2- 172.21.15.32:0/1677493507 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2d0c107420 0x7f2d0c1989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:42.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:42.586+0000 7f2d11c28700 1 --2- 172.21.15.32:0/1677493507 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2d0c1026f0 0x7f2d0c19e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:42.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:42.586+0000 7f2d11c28700 1 --2- 172.21.15.32:0/1677493507 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d0c1030b0 0x7f2d0c19ed60 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:42.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:42.586+0000 7f2d11c28700 1 -- 172.21.15.32:0/1677493507 >> 172.21.15.32:0/1677493507 conn(0x7f2d0c078210 msgr2=0x7f2d0c1007a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:46:42.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:42.586+0000 7f2d11c28700 1 -- 172.21.15.32:0/1677493507 shutdown_connections 2024-05-03T18:46:42.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:42.586+0000 7f2d11c28700 1 -- 172.21.15.32:0/1677493507 wait complete. 2024-05-03T18:46:42.590 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2024-05-03T18:46:43.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:46:43 smithi032 ceph-mon[29763]: pgmap v338: 97 pgs: 97 active+clean; 631 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:46:43.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:46:43 smithi032 ceph-mon[29763]: from='client.14934 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-03T18:46:43.379 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":338,"stamp":"2024-05-03T18:46:41.992524+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":646436,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":663,"num_read_kb":416,"num_write":284,"num_write_kb":2253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":2951957,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":230,"ondisk_log_size":230,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":374024,"kb_used_data":12324,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124567800,"statfs":{"total":1151940427776,"available":1151557427200,"internally_reserved":0,"allocated":12619776,"data_stored":7036068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":370193723},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":70,"num_read_kb":35,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003561"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.585847+0000","last_change":"2024-05-03T18:44:49.430467+0000","last_active":"2024-05-03T18:45:47.585847+0000","last_peered":"2024-05-03T18:45:47.585847+0000","last_clean":"2024-05-03T18:45:47.585847+0000","last_became_active":"2024-05-03T18:44:49.430147+0000","last_became_peered":"2024-05-03T18:44:49.430147+0000","last_unstale":"2024-05-03T18:45:47.585847+0000","last_undegraded":"2024-05-03T18:45:47.585847+0000","last_fullsized":"2024-05-03T18:45:47.585847+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:49:22.725690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.586166+0000","last_change":"2024-05-03T18:44:49.432057+0000","last_active":"2024-05-03T18:45:47.586166+0000","last_peered":"2024-05-03T18:45:47.586166+0000","last_clean":"2024-05-03T18:45:47.586166+0000","last_became_active":"2024-05-03T18:44:49.430867+0000","last_became_peered":"2024-05-03T18:44:49.430867+0000","last_unstale":"2024-05-03T18:45:47.586166+0000","last_undegraded":"2024-05-03T18:45:47.586166+0000","last_fullsized":"2024-05-03T18:45:47.586166+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:31:16.470041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.586095+0000","last_change":"2024-05-03T18:44:49.433622+0000","last_active":"2024-05-03T18:45:47.586095+0000","last_peered":"2024-05-03T18:45:47.586095+0000","last_clean":"2024-05-03T18:45:47.586095+0000","last_became_active":"2024-05-03T18:44:49.431803+0000","last_became_peered":"2024-05-03T18:44:49.431803+0000","last_unstale":"2024-05-03T18:45:47.586095+0000","last_undegraded":"2024-05-03T18:45:47.586095+0000","last_fullsized":"2024-05-03T18:45:47.586095+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:40:18.492498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.635138+0000","last_change":"2024-05-03T18:44:49.437469+0000","last_active":"2024-05-03T18:45:47.635138+0000","last_peered":"2024-05-03T18:45:47.635138+0000","last_clean":"2024-05-03T18:45:47.635138+0000","last_became_active":"2024-05-03T18:44:49.435519+0000","last_became_peered":"2024-05-03T18:44:49.435519+0000","last_unstale":"2024-05-03T18:45:47.635138+0000","last_undegraded":"2024-05-03T18:45:47.635138+0000","last_fullsized":"2024-05-03T18:45:47.635138+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:11:07.246146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.958037+0000","last_change":"2024-05-03T18:45:47.958037+0000","last_active":"2024-05-03T18:45:47.958037+0000","last_peered":"2024-05-03T18:45:47.958037+0000","last_clean":"2024-05-03T18:45:47.958037+0000","last_became_active":"2024-05-03T18:45:47.957558+0000","last_became_peered":"2024-05-03T18:45:47.957558+0000","last_unstale":"2024-05-03T18:45:47.958037+0000","last_undegraded":"2024-05-03T18:45:47.958037+0000","last_fullsized":"2024-05-03T18:45:47.958037+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:19:31.077969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.589949+0000","last_change":"2024-05-03T18:45:47.589949+0000","last_active":"2024-05-03T18:45:47.589949+0000","last_peered":"2024-05-03T18:45:47.589949+0000","last_clean":"2024-05-03T18:45:47.589949+0000","last_became_active":"2024-05-03T18:45:47.589517+0000","last_became_peered":"2024-05-03T18:45:47.589517+0000","last_unstale":"2024-05-03T18:45:47.589949+0000","last_undegraded":"2024-05-03T18:45:47.589949+0000","last_fullsized":"2024-05-03T18:45:47.589949+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:00:41.296563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.589866+0000","last_change":"2024-05-03T18:45:47.589866+0000","last_active":"2024-05-03T18:45:47.589866+0000","last_peered":"2024-05-03T18:45:47.589866+0000","last_clean":"2024-05-03T18:45:47.589866+0000","last_became_active":"2024-05-03T18:45:47.589402+0000","last_became_peered":"2024-05-03T18:45:47.589402+0000","last_unstale":"2024-05-03T18:45:47.589866+0000","last_undegraded":"2024-05-03T18:45:47.589866+0000","last_fullsized":"2024-05-03T18:45:47.589866+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:55:48.047379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,8],"acting":[7,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.582342+0000","last_change":"2024-05-03T18:44:49.432804+0000","last_active":"2024-05-03T18:45:47.582342+0000","last_peered":"2024-05-03T18:45:47.582342+0000","last_clean":"2024-05-03T18:45:47.582342+0000","last_became_active":"2024-05-03T18:44:49.431828+0000","last_became_peered":"2024-05-03T18:44:49.431828+0000","last_unstale":"2024-05-03T18:45:47.582342+0000","last_undegraded":"2024-05-03T18:45:47.582342+0000","last_fullsized":"2024-05-03T18:45:47.582342+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:06:40.810972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.583535+0000","last_change":"2024-05-03T18:44:49.428919+0000","last_active":"2024-05-03T18:45:47.583535+0000","last_peered":"2024-05-03T18:45:47.583535+0000","last_clean":"2024-05-03T18:45:47.583535+0000","last_became_active":"2024-05-03T18:44:49.428350+0000","last_became_peered":"2024-05-03T18:44:49.428350+0000","last_unstale":"2024-05-03T18:45:47.583535+0000","last_undegraded":"2024-05-03T18:45:47.583535+0000","last_fullsized":"2024-05-03T18:45:47.583535+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T18:50:09.895485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.587062+0000","last_change":"2024-05-03T18:44:49.431765+0000","last_active":"2024-05-03T18:45:47.587062+0000","last_peered":"2024-05-03T18:45:47.587062+0000","last_clean":"2024-05-03T18:45:47.587062+0000","last_became_active":"2024-05-03T18:44:49.431417+0000","last_became_peered":"2024-05-03T18:44:49.431417+0000","last_unstale":"2024-05-03T18:45:47.587062+0000","last_undegraded":"2024-05-03T18:45:47.587062+0000","last_fullsized":"2024-05-03T18:45:47.587062+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:24:58.097954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.584833+0000","last_change":"2024-05-03T18:44:49.434154+0000","last_active":"2024-05-03T18:45:47.584833+0000","last_peered":"2024-05-03T18:45:47.584833+0000","last_clean":"2024-05-03T18:45:47.584833+0000","last_became_active":"2024-05-03T18:44:49.433806+0000","last_became_peered":"2024-05-03T18:44:49.433806+0000","last_unstale":"2024-05-03T18:45:47.584833+0000","last_undegraded":"2024-05-03T18:45:47.584833+0000","last_fullsized":"2024-05-03T18:45:47.584833+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:52:39.233556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.635100+0000","last_change":"2024-05-03T18:44:49.435385+0000","last_active":"2024-05-03T18:45:47.635100+0000","last_peered":"2024-05-03T18:45:47.635100+0000","last_clean":"2024-05-03T18:45:47.635100+0000","last_became_active":"2024-05-03T18:44:49.434727+0000","last_became_peered":"2024-05-03T18:44:49.434727+0000","last_unstale":"2024-05-03T18:45:47.635100+0000","last_undegraded":"2024-05-03T18:45:47.635100+0000","last_fullsized":"2024-05-03T18:45:47.635100+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:08:45.986371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.956820+0000","last_change":"2024-05-03T18:44:49.428536+0000","last_active":"2024-05-03T18:45:47.956820+0000","last_peered":"2024-05-03T18:45:47.956820+0000","last_clean":"2024-05-03T18:45:47.956820+0000","last_became_active":"2024-05-03T18:44:49.427630+0000","last_became_peered":"2024-05-03T18:44:49.427630+0000","last_unstale":"2024-05-03T18:45:47.956820+0000","last_undegraded":"2024-05-03T18:45:47.956820+0000","last_fullsized":"2024-05-03T18:45:47.956820+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:24:09.622403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.585815+0000","last_change":"2024-05-03T18:44:49.429236+0000","last_active":"2024-05-03T18:45:47.585815+0000","last_peered":"2024-05-03T18:45:47.585815+0000","last_clean":"2024-05-03T18:45:47.585815+0000","last_became_active":"2024-05-03T18:44:49.428942+0000","last_became_peered":"2024-05-03T18:44:49.428942+0000","last_unstale":"2024-05-03T18:45:47.585815+0000","last_undegraded":"2024-05-03T18:45:47.585815+0000","last_fullsized":"2024-05-03T18:45:47.585815+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:16:44.241736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.585732+0000","last_change":"2024-05-03T18:44:49.433388+0000","last_active":"2024-05-03T18:45:47.585732+0000","last_peered":"2024-05-03T18:45:47.585732+0000","last_clean":"2024-05-03T18:45:47.585732+0000","last_became_active":"2024-05-03T18:44:49.431869+0000","last_became_peered":"2024-05-03T18:44:49.431869+0000","last_unstale":"2024-05-03T18:45:47.585732+0000","last_undegraded":"2024-05-03T18:45:47.585732+0000","last_fullsized":"2024-05-03T18:45:47.585732+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:36:52.803391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.582873+0000","last_change":"2024-05-03T18:44:49.425781+0000","last_active":"2024-05-03T18:45:47.582873+0000","last_peered":"2024-05-03T18:45:47.582873+0000","last_clean":"2024-05-03T18:45:47.582873+0000","last_became_active":"2024-05-03T18:44:49.425248+0000","last_became_peered":"2024-05-03T18:44:49.425248+0000","last_unstale":"2024-05-03T18:45:47.582873+0000","last_undegraded":"2024-05-03T18:45:47.582873+0000","last_fullsized":"2024-05-03T18:45:47.582873+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:27:25.017459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.589399+0000","last_change":"2024-05-03T18:45:47.589399+0000","last_active":"2024-05-03T18:45:47.589399+0000","last_peered":"2024-05-03T18:45:47.589399+0000","last_clean":"2024-05-03T18:45:47.589399+0000","last_became_active":"2024-05-03T18:45:47.589028+0000","last_became_peered":"2024-05-03T18:45:47.589028+0000","last_unstale":"2024-05-03T18:45:47.589399+0000","last_undegraded":"2024-05-03T18:45:47.589399+0000","last_fullsized":"2024-05-03T18:45:47.589399+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:00:47.913452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,0],"acting":[11,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.584936+0000","last_change":"2024-05-03T18:44:49.427176+0000","last_active":"2024-05-03T18:45:47.584936+0000","last_peered":"2024-05-03T18:45:47.584936+0000","last_clean":"2024-05-03T18:45:47.584936+0000","last_became_active":"2024-05-03T18:44:49.426546+0000","last_became_peered":"2024-05-03T18:44:49.426546+0000","last_unstale":"2024-05-03T18:45:47.584936+0000","last_undegraded":"2024-05-03T18:45:47.584936+0000","last_fullsized":"2024-05-03T18:45:47.584936+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:29:22.343376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.589214+0000","last_change":"2024-05-03T18:44:49.435037+0000","last_active":"2024-05-03T18:45:47.589214+0000","last_peered":"2024-05-03T18:45:47.589214+0000","last_clean":"2024-05-03T18:45:47.589214+0000","last_became_active":"2024-05-03T18:44:49.434693+0000","last_became_peered":"2024-05-03T18:44:49.434693+0000","last_unstale":"2024-05-03T18:45:47.589214+0000","last_undegraded":"2024-05-03T18:45:47.589214+0000","last_fullsized":"2024-05-03T18:45:47.589214+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:26:33.923608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"78'9","reported_seq":35,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.585020+0000","last_change":"2024-05-03T18:44:46.402606+0000","last_active":"2024-05-03T18:45:47.585020+0000","last_peered":"2024-05-03T18:45:47.585020+0000","last_clean":"2024-05-03T18:45:47.585020+0000","last_became_active":"2024-05-03T18:44:46.402245+0000","last_became_peered":"2024-05-03T18:44:46.402245+0000","last_unstale":"2024-05-03T18:45:47.585020+0000","last_undegraded":"2024-05-03T18:45:47.585020+0000","last_fullsized":"2024-05-03T18:45:47.585020+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_clean_scrub_stamp":"2024-05-03T18:44:45.380544+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:29:33.726119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.582919+0000","last_change":"2024-05-03T18:44:49.435279+0000","last_active":"2024-05-03T18:45:47.582919+0000","last_peered":"2024-05-03T18:45:47.582919+0000","last_clean":"2024-05-03T18:45:47.582919+0000","last_became_active":"2024-05-03T18:44:49.434779+0000","last_became_peered":"2024-05-03T18:44:49.434779+0000","last_unstale":"2024-05-03T18:45:47.582919+0000","last_undegraded":"2024-05-03T18:45:47.582919+0000","last_fullsized":"2024-05-03T18:45:47.582919+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:56:56.762051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"78'2","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.582807+0000","last_change":"2024-05-03T18:44:46.644899+0000","last_active":"2024-05-03T18:45:47.582807+0000","last_peered":"2024-05-03T18:45:47.582807+0000","last_clean":"2024-05-03T18:45:47.582807+0000","last_became_active":"2024-05-03T18:44:46.644508+0000","last_became_peered":"2024-05-03T18:44:46.644508+0000","last_unstale":"2024-05-03T18:45:47.582807+0000","last_undegraded":"2024-05-03T18:45:47.582807+0000","last_fullsized":"2024-05-03T18:45:47.582807+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_clean_scrub_stamp":"2024-05-03T18:44:45.380544+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T06:38:09.379634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.586375+0000","last_change":"2024-05-03T18:44:49.429720+0000","last_active":"2024-05-03T18:45:47.586375+0000","last_peered":"2024-05-03T18:45:47.586375+0000","last_clean":"2024-05-03T18:45:47.586375+0000","last_became_active":"2024-05-03T18:44:49.428399+0000","last_became_peered":"2024-05-03T18:44:49.428399+0000","last_unstale":"2024-05-03T18:45:47.586375+0000","last_undegraded":"2024-05-03T18:45:47.586375+0000","last_fullsized":"2024-05-03T18:45:47.586375+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:17:57.325926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"78'4","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.585620+0000","last_change":"2024-05-03T18:44:46.410397+0000","last_active":"2024-05-03T18:45:47.585620+0000","last_peered":"2024-05-03T18:45:47.585620+0000","last_clean":"2024-05-03T18:45:47.585620+0000","last_became_active":"2024-05-03T18:44:46.410030+0000","last_became_peered":"2024-05-03T18:44:46.410030+0000","last_unstale":"2024-05-03T18:45:47.585620+0000","last_undegraded":"2024-05-03T18:45:47.585620+0000","last_fullsized":"2024-05-03T18:45:47.585620+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_clean_scrub_stamp":"2024-05-03T18:44:45.380544+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:15:26.130070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.583088+0000","last_change":"2024-05-03T18:44:49.430335+0000","last_active":"2024-05-03T18:45:47.583088+0000","last_peered":"2024-05-03T18:45:47.583088+0000","last_clean":"2024-05-03T18:45:47.583088+0000","last_became_active":"2024-05-03T18:44:49.429991+0000","last_became_peered":"2024-05-03T18:44:49.429991+0000","last_unstale":"2024-05-03T18:45:47.583088+0000","last_undegraded":"2024-05-03T18:45:47.583088+0000","last_fullsized":"2024-05-03T18:45:47.583088+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:06:28.718138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"78'2","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.585877+0000","last_change":"2024-05-03T18:44:46.406552+0000","last_active":"2024-05-03T18:45:47.585877+0000","last_peered":"2024-05-03T18:45:47.585877+0000","last_clean":"2024-05-03T18:45:47.585877+0000","last_became_active":"2024-05-03T18:44:46.406160+0000","last_became_peered":"2024-05-03T18:44:46.406160+0000","last_unstale":"2024-05-03T18:45:47.585877+0000","last_undegraded":"2024-05-03T18:45:47.585877+0000","last_fullsized":"2024-05-03T18:45:47.585877+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_clean_scrub_stamp":"2024-05-03T18:44:45.380544+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:53:13.769225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.584609+0000","last_change":"2024-05-03T18:44:49.441301+0000","last_active":"2024-05-03T18:45:47.584609+0000","last_peered":"2024-05-03T18:45:47.584609+0000","last_clean":"2024-05-03T18:45:47.584609+0000","last_became_active":"2024-05-03T18:44:49.440936+0000","last_became_peered":"2024-05-03T18:44:49.440936+0000","last_unstale":"2024-05-03T18:45:47.584609+0000","last_undegraded":"2024-05-03T18:45:47.584609+0000","last_fullsized":"2024-05-03T18:45:47.584609+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:55:41.257325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"78'6","reported_seq":130,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:46:39.623881+0000","last_change":"2024-05-03T18:44:46.404827+0000","last_active":"2024-05-03T18:46:39.623881+0000","last_peered":"2024-05-03T18:46:39.623881+0000","last_clean":"2024-05-03T18:46:39.623881+0000","last_became_active":"2024-05-03T18:44:46.404455+0000","last_became_peered":"2024-05-03T18:44:46.404455+0000","last_unstale":"2024-05-03T18:46:39.623881+0000","last_undegraded":"2024-05-03T18:46:39.623881+0000","last_fullsized":"2024-05-03T18:46:39.623881+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_clean_scrub_stamp":"2024-05-03T18:44:45.380544+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:23:32.855160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":99,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.585568+0000","last_change":"2024-05-03T18:44:49.433656+0000","last_active":"2024-05-03T18:45:47.585568+0000","last_peered":"2024-05-03T18:45:47.585568+0000","last_clean":"2024-05-03T18:45:47.585568+0000","last_became_active":"2024-05-03T18:44:49.433326+0000","last_became_peered":"2024-05-03T18:44:49.433326+0000","last_unstale":"2024-05-03T18:45:47.585568+0000","last_undegraded":"2024-05-03T18:45:47.585568+0000","last_fullsized":"2024-05-03T18:45:47.585568+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:44:56.937334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"78'3","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.586938+0000","last_change":"2024-05-03T18:44:46.403960+0000","last_active":"2024-05-03T18:45:47.586938+0000","last_peered":"2024-05-03T18:45:47.586938+0000","last_clean":"2024-05-03T18:45:47.586938+0000","last_became_active":"2024-05-03T18:44:46.403560+0000","last_became_peered":"2024-05-03T18:44:46.403560+0000","last_unstale":"2024-05-03T18:45:47.586938+0000","last_undegraded":"2024-05-03T18:45:47.586938+0000","last_fullsized":"2024-05-03T18:45:47.586938+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_clean_scrub_stamp":"2024-05-03T18:44:45.380544+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:58:32.823640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.957785+0000","last_change":"2024-05-03T18:45:47.957785+0000","last_active":"2024-05-03T18:45:47.957785+0000","last_peered":"2024-05-03T18:45:47.957785+0000","last_clean":"2024-05-03T18:45:47.957785+0000","last_became_active":"2024-05-03T18:45:47.957429+0000","last_became_peered":"2024-05-03T18:45:47.957429+0000","last_unstale":"2024-05-03T18:45:47.957785+0000","last_undegraded":"2024-05-03T18:45:47.957785+0000","last_fullsized":"2024-05-03T18:45:47.957785+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T18:47:27.922032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,3],"acting":[5,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"78'2","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.956500+0000","last_change":"2024-05-03T18:44:46.400282+0000","last_active":"2024-05-03T18:45:47.956500+0000","last_peered":"2024-05-03T18:45:47.956500+0000","last_clean":"2024-05-03T18:45:47.956500+0000","last_became_active":"2024-05-03T18:44:46.399873+0000","last_became_peered":"2024-05-03T18:44:46.399873+0000","last_unstale":"2024-05-03T18:45:47.956500+0000","last_undegraded":"2024-05-03T18:45:47.956500+0000","last_fullsized":"2024-05-03T18:45:47.956500+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_clean_scrub_stamp":"2024-05-03T18:44:45.380544+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:12:10.005145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.582371+0000","last_change":"2024-05-03T18:44:49.422311+0000","last_active":"2024-05-03T18:45:47.582371+0000","last_peered":"2024-05-03T18:45:47.582371+0000","last_clean":"2024-05-03T18:45:47.582371+0000","last_became_active":"2024-05-03T18:44:49.421962+0000","last_became_peered":"2024-05-03T18:44:49.421962+0000","last_unstale":"2024-05-03T18:45:47.582371+0000","last_undegraded":"2024-05-03T18:45:47.582371+0000","last_fullsized":"2024-05-03T18:45:47.582371+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:50:46.407989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"78'1","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.583898+0000","last_change":"2024-05-03T18:44:46.410969+0000","last_active":"2024-05-03T18:45:47.583898+0000","last_peered":"2024-05-03T18:45:47.583898+0000","last_clean":"2024-05-03T18:45:47.583898+0000","last_became_active":"2024-05-03T18:44:46.410089+0000","last_became_peered":"2024-05-03T18:44:46.410089+0000","last_unstale":"2024-05-03T18:45:47.583898+0000","last_undegraded":"2024-05-03T18:45:47.583898+0000","last_fullsized":"2024-05-03T18:45:47.583898+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_clean_scrub_stamp":"2024-05-03T18:44:45.380544+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:13:51.641235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.584266+0000","last_change":"2024-05-03T18:44:49.426946+0000","last_active":"2024-05-03T18:45:47.584266+0000","last_peered":"2024-05-03T18:45:47.584266+0000","last_clean":"2024-05-03T18:45:47.584266+0000","last_became_active":"2024-05-03T18:44:49.426609+0000","last_became_peered":"2024-05-03T18:44:49.426609+0000","last_unstale":"2024-05-03T18:45:47.584266+0000","last_undegraded":"2024-05-03T18:45:47.584266+0000","last_fullsized":"2024-05-03T18:45:47.584266+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:34:59.944769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"78'3","reported_seq":120,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:46:35.617295+0000","last_change":"2024-05-03T18:44:46.405415+0000","last_active":"2024-05-03T18:46:35.617295+0000","last_peered":"2024-05-03T18:46:35.617295+0000","last_clean":"2024-05-03T18:46:35.617295+0000","last_became_active":"2024-05-03T18:44:46.404928+0000","last_became_peered":"2024-05-03T18:44:46.404928+0000","last_unstale":"2024-05-03T18:46:35.617295+0000","last_undegraded":"2024-05-03T18:46:35.617295+0000","last_fullsized":"2024-05-03T18:46:35.617295+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_clean_scrub_stamp":"2024-05-03T18:44:45.380544+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:33:10.979432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":92,"num_read_kb":92,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.583489+0000","last_change":"2024-05-03T18:44:49.427873+0000","last_active":"2024-05-03T18:45:47.583489+0000","last_peered":"2024-05-03T18:45:47.583489+0000","last_clean":"2024-05-03T18:45:47.583489+0000","last_became_active":"2024-05-03T18:44:49.426797+0000","last_became_peered":"2024-05-03T18:44:49.426797+0000","last_unstale":"2024-05-03T18:45:47.583489+0000","last_undegraded":"2024-05-03T18:45:47.583489+0000","last_fullsized":"2024-05-03T18:45:47.583489+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:07:28.989454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"78'2","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.584594+0000","last_change":"2024-05-03T18:44:46.407167+0000","last_active":"2024-05-03T18:45:47.584594+0000","last_peered":"2024-05-03T18:45:47.584594+0000","last_clean":"2024-05-03T18:45:47.584594+0000","last_became_active":"2024-05-03T18:44:46.406787+0000","last_became_peered":"2024-05-03T18:44:46.406787+0000","last_unstale":"2024-05-03T18:45:47.584594+0000","last_undegraded":"2024-05-03T18:45:47.584594+0000","last_fullsized":"2024-05-03T18:45:47.584594+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_clean_scrub_stamp":"2024-05-03T18:44:45.380544+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:33:12.854056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.957910+0000","last_change":"2024-05-03T18:45:47.957910+0000","last_active":"2024-05-03T18:45:47.957910+0000","last_peered":"2024-05-03T18:45:47.957910+0000","last_clean":"2024-05-03T18:45:47.957910+0000","last_became_active":"2024-05-03T18:45:47.957452+0000","last_became_peered":"2024-05-03T18:45:47.957452+0000","last_unstale":"2024-05-03T18:45:47.957910+0000","last_undegraded":"2024-05-03T18:45:47.957910+0000","last_fullsized":"2024-05-03T18:45:47.957910+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:22:03.503320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"78'4","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.584723+0000","last_change":"2024-05-03T18:44:46.647273+0000","last_active":"2024-05-03T18:45:47.584723+0000","last_peered":"2024-05-03T18:45:47.584723+0000","last_clean":"2024-05-03T18:45:47.584723+0000","last_became_active":"2024-05-03T18:44:46.646881+0000","last_became_peered":"2024-05-03T18:44:46.646881+0000","last_unstale":"2024-05-03T18:45:47.584723+0000","last_undegraded":"2024-05-03T18:45:47.584723+0000","last_fullsized":"2024-05-03T18:45:47.584723+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_clean_scrub_stamp":"2024-05-03T18:44:45.380544+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:03:47.988180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.583137+0000","last_change":"2024-05-03T18:44:49.436525+0000","last_active":"2024-05-03T18:45:47.583137+0000","last_peered":"2024-05-03T18:45:47.583137+0000","last_clean":"2024-05-03T18:45:47.583137+0000","last_became_active":"2024-05-03T18:44:49.436109+0000","last_became_peered":"2024-05-03T18:44:49.436109+0000","last_unstale":"2024-05-03T18:45:47.583137+0000","last_undegraded":"2024-05-03T18:45:47.583137+0000","last_fullsized":"2024-05-03T18:45:47.583137+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:08:42.239248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"78'27","reported_seq":244,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:46:36.618582+0000","last_change":"2024-05-03T18:44:46.407936+0000","last_active":"2024-05-03T18:46:36.618582+0000","last_peered":"2024-05-03T18:46:36.618582+0000","last_clean":"2024-05-03T18:46:36.618582+0000","last_became_active":"2024-05-03T18:44:46.407504+0000","last_became_peered":"2024-05-03T18:44:46.407504+0000","last_unstale":"2024-05-03T18:46:36.618582+0000","last_undegraded":"2024-05-03T18:46:36.618582+0000","last_fullsized":"2024-05-03T18:46:36.618582+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_clean_scrub_stamp":"2024-05-03T18:44:45.380544+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:26:27.213572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":51833,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":192,"num_read_kb":146,"num_write":28,"num_write_kb":60,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.585324+0000","last_change":"2024-05-03T18:44:49.438509+0000","last_active":"2024-05-03T18:45:47.585324+0000","last_peered":"2024-05-03T18:45:47.585324+0000","last_clean":"2024-05-03T18:45:47.585324+0000","last_became_active":"2024-05-03T18:44:49.438185+0000","last_became_peered":"2024-05-03T18:44:49.438185+0000","last_unstale":"2024-05-03T18:45:47.585324+0000","last_undegraded":"2024-05-03T18:45:47.585324+0000","last_fullsized":"2024-05-03T18:45:47.585324+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:57:28.072095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"78'6","reported_seq":33,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.589344+0000","last_change":"2024-05-03T18:44:46.407842+0000","last_active":"2024-05-03T18:45:47.589344+0000","last_peered":"2024-05-03T18:45:47.589344+0000","last_clean":"2024-05-03T18:45:47.589344+0000","last_became_active":"2024-05-03T18:44:46.407506+0000","last_became_peered":"2024-05-03T18:44:46.407506+0000","last_unstale":"2024-05-03T18:45:47.589344+0000","last_undegraded":"2024-05-03T18:45:47.589344+0000","last_fullsized":"2024-05-03T18:45:47.589344+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_clean_scrub_stamp":"2024-05-03T18:44:45.380544+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:51:33.101539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.956231+0000","last_change":"2024-05-03T18:44:49.433214+0000","last_active":"2024-05-03T18:45:47.956231+0000","last_peered":"2024-05-03T18:45:47.956231+0000","last_clean":"2024-05-03T18:45:47.956231+0000","last_became_active":"2024-05-03T18:44:49.432879+0000","last_became_peered":"2024-05-03T18:44:49.432879+0000","last_unstale":"2024-05-03T18:45:47.956231+0000","last_undegraded":"2024-05-03T18:45:47.956231+0000","last_fullsized":"2024-05-03T18:45:47.956231+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:35:17.920304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"78'3","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.583880+0000","last_change":"2024-05-03T18:44:46.405280+0000","last_active":"2024-05-03T18:45:47.583880+0000","last_peered":"2024-05-03T18:45:47.583880+0000","last_clean":"2024-05-03T18:45:47.583880+0000","last_became_active":"2024-05-03T18:44:46.404126+0000","last_became_peered":"2024-05-03T18:44:46.404126+0000","last_unstale":"2024-05-03T18:45:47.583880+0000","last_undegraded":"2024-05-03T18:45:47.583880+0000","last_fullsized":"2024-05-03T18:45:47.583880+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_clean_scrub_stamp":"2024-05-03T18:44:45.380544+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:14:52.146440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.585033+0000","last_change":"2024-05-03T18:44:49.430582+0000","last_active":"2024-05-03T18:45:47.585033+0000","last_peered":"2024-05-03T18:45:47.585033+0000","last_clean":"2024-05-03T18:45:47.585033+0000","last_became_active":"2024-05-03T18:44:49.430246+0000","last_became_peered":"2024-05-03T18:44:49.430246+0000","last_unstale":"2024-05-03T18:45:47.585033+0000","last_undegraded":"2024-05-03T18:45:47.585033+0000","last_fullsized":"2024-05-03T18:45:47.585033+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:17:13.504586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"78'3","reported_seq":123,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:46:36.618681+0000","last_change":"2024-05-03T18:44:46.404276+0000","last_active":"2024-05-03T18:46:36.618681+0000","last_peered":"2024-05-03T18:46:36.618681+0000","last_clean":"2024-05-03T18:46:36.618681+0000","last_became_active":"2024-05-03T18:44:46.403843+0000","last_became_peered":"2024-05-03T18:44:46.403843+0000","last_unstale":"2024-05-03T18:46:36.618681+0000","last_undegraded":"2024-05-03T18:46:36.618681+0000","last_fullsized":"2024-05-03T18:46:36.618681+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_clean_scrub_stamp":"2024-05-03T18:44:45.380544+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:05:49.483444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.584250+0000","last_change":"2024-05-03T18:44:49.427950+0000","last_active":"2024-05-03T18:45:47.584250+0000","last_peered":"2024-05-03T18:45:47.584250+0000","last_clean":"2024-05-03T18:45:47.584250+0000","last_became_active":"2024-05-03T18:44:49.427469+0000","last_became_peered":"2024-05-03T18:44:49.427469+0000","last_unstale":"2024-05-03T18:45:47.584250+0000","last_undegraded":"2024-05-03T18:45:47.584250+0000","last_fullsized":"2024-05-03T18:45:47.584250+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:33:35.690809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"78'15","reported_seq":41,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.585236+0000","last_change":"2024-05-03T18:44:46.645189+0000","last_active":"2024-05-03T18:45:47.585236+0000","last_peered":"2024-05-03T18:45:47.585236+0000","last_clean":"2024-05-03T18:45:47.585236+0000","last_became_active":"2024-05-03T18:44:46.644642+0000","last_became_peered":"2024-05-03T18:44:46.644642+0000","last_unstale":"2024-05-03T18:45:47.585236+0000","last_undegraded":"2024-05-03T18:45:47.585236+0000","last_fullsized":"2024-05-03T18:45:47.585236+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_clean_scrub_stamp":"2024-05-03T18:44:45.380544+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:16:49.003568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.584408+0000","last_change":"2024-05-03T18:44:49.438931+0000","last_active":"2024-05-03T18:45:47.584408+0000","last_peered":"2024-05-03T18:45:47.584408+0000","last_clean":"2024-05-03T18:45:47.584408+0000","last_became_active":"2024-05-03T18:44:49.438573+0000","last_became_peered":"2024-05-03T18:44:49.438573+0000","last_unstale":"2024-05-03T18:45:47.584408+0000","last_undegraded":"2024-05-03T18:45:47.584408+0000","last_fullsized":"2024-05-03T18:45:47.584408+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T06:09:31.042649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"78'2","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.585857+0000","last_change":"2024-05-03T18:44:46.406752+0000","last_active":"2024-05-03T18:45:47.585857+0000","last_peered":"2024-05-03T18:45:47.585857+0000","last_clean":"2024-05-03T18:45:47.585857+0000","last_became_active":"2024-05-03T18:44:46.406288+0000","last_became_peered":"2024-05-03T18:44:46.406288+0000","last_unstale":"2024-05-03T18:45:47.585857+0000","last_undegraded":"2024-05-03T18:45:47.585857+0000","last_fullsized":"2024-05-03T18:45:47.585857+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_clean_scrub_stamp":"2024-05-03T18:44:45.380544+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:20:33.947006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.588049+0000","last_change":"2024-05-03T18:45:47.588049+0000","last_active":"2024-05-03T18:45:47.588049+0000","last_peered":"2024-05-03T18:45:47.588049+0000","last_clean":"2024-05-03T18:45:47.588049+0000","last_became_active":"2024-05-03T18:45:47.587534+0000","last_became_peered":"2024-05-03T18:45:47.587534+0000","last_unstale":"2024-05-03T18:45:47.588049+0000","last_undegraded":"2024-05-03T18:45:47.588049+0000","last_fullsized":"2024-05-03T18:45:47.588049+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:16:36.601533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7],"acting":[11,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.584305+0000","last_change":"2024-05-03T18:44:46.647155+0000","last_active":"2024-05-03T18:45:47.584305+0000","last_peered":"2024-05-03T18:45:47.584305+0000","last_clean":"2024-05-03T18:45:47.584305+0000","last_became_active":"2024-05-03T18:44:46.646719+0000","last_became_peered":"2024-05-03T18:44:46.646719+0000","last_unstale":"2024-05-03T18:45:47.584305+0000","last_undegraded":"2024-05-03T18:45:47.584305+0000","last_fullsized":"2024-05-03T18:45:47.584305+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_clean_scrub_stamp":"2024-05-03T18:44:45.380544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T18:49:13.701498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"78'1","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.586646+0000","last_change":"2024-05-03T18:44:49.423776+0000","last_active":"2024-05-03T18:45:47.586646+0000","last_peered":"2024-05-03T18:45:47.586646+0000","last_clean":"2024-05-03T18:45:47.586646+0000","last_became_active":"2024-05-03T18:44:49.423414+0000","last_became_peered":"2024-05-03T18:44:49.423414+0000","last_unstale":"2024-05-03T18:45:47.586646+0000","last_undegraded":"2024-05-03T18:45:47.586646+0000","last_fullsized":"2024-05-03T18:45:47.586646+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:48:03.556270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"78'1","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.584512+0000","last_change":"2024-05-03T18:44:46.407725+0000","last_active":"2024-05-03T18:45:47.584512+0000","last_peered":"2024-05-03T18:45:47.584512+0000","last_clean":"2024-05-03T18:45:47.584512+0000","last_became_active":"2024-05-03T18:44:46.407317+0000","last_became_peered":"2024-05-03T18:44:46.407317+0000","last_unstale":"2024-05-03T18:45:47.584512+0000","last_undegraded":"2024-05-03T18:45:47.584512+0000","last_fullsized":"2024-05-03T18:45:47.584512+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_clean_scrub_stamp":"2024-05-03T18:44:45.380544+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:02:32.739918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.586698+0000","last_change":"2024-05-03T18:44:49.432439+0000","last_active":"2024-05-03T18:45:47.586698+0000","last_peered":"2024-05-03T18:45:47.586698+0000","last_clean":"2024-05-03T18:45:47.586698+0000","last_became_active":"2024-05-03T18:44:49.431792+0000","last_became_peered":"2024-05-03T18:44:49.431792+0000","last_unstale":"2024-05-03T18:45:47.586698+0000","last_undegraded":"2024-05-03T18:45:47.586698+0000","last_fullsized":"2024-05-03T18:45:47.586698+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:12:46.223514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.584147+0000","last_change":"2024-05-03T18:44:49.440075+0000","last_active":"2024-05-03T18:45:47.584147+0000","last_peered":"2024-05-03T18:45:47.584147+0000","last_clean":"2024-05-03T18:45:47.584147+0000","last_became_active":"2024-05-03T18:44:49.439655+0000","last_became_peered":"2024-05-03T18:44:49.439655+0000","last_unstale":"2024-05-03T18:45:47.584147+0000","last_undegraded":"2024-05-03T18:45:47.584147+0000","last_fullsized":"2024-05-03T18:45:47.584147+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:19:03.306588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"78'4","reported_seq":125,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:46:38.621124+0000","last_change":"2024-05-03T18:44:46.645090+0000","last_active":"2024-05-03T18:46:38.621124+0000","last_peered":"2024-05-03T18:46:38.621124+0000","last_clean":"2024-05-03T18:46:38.621124+0000","last_became_active":"2024-05-03T18:44:46.644722+0000","last_became_peered":"2024-05-03T18:44:46.644722+0000","last_unstale":"2024-05-03T18:46:38.621124+0000","last_undegraded":"2024-05-03T18:46:38.621124+0000","last_fullsized":"2024-05-03T18:46:38.621124+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_clean_scrub_stamp":"2024-05-03T18:44:45.380544+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T20:44:38.032522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":95,"num_read_kb":95,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.586922+0000","last_change":"2024-05-03T18:45:47.586554+0000","last_active":"2024-05-03T18:45:47.586922+0000","last_peered":"2024-05-03T18:45:47.586922+0000","last_clean":"2024-05-03T18:45:47.586922+0000","last_became_active":"2024-05-03T18:45:47.586247+0000","last_became_peered":"2024-05-03T18:45:47.586247+0000","last_unstale":"2024-05-03T18:45:47.586922+0000","last_undegraded":"2024-05-03T18:45:47.586922+0000","last_fullsized":"2024-05-03T18:45:47.586922+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T06:30:16.743598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.3","version":"78'3","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.635251+0000","last_change":"2024-05-03T18:44:46.402866+0000","last_active":"2024-05-03T18:45:47.635251+0000","last_peered":"2024-05-03T18:45:47.635251+0000","last_clean":"2024-05-03T18:45:47.635251+0000","last_became_active":"2024-05-03T18:44:46.402394+0000","last_became_peered":"2024-05-03T18:44:46.402394+0000","last_unstale":"2024-05-03T18:45:47.635251+0000","last_undegraded":"2024-05-03T18:45:47.635251+0000","last_fullsized":"2024-05-03T18:45:47.635251+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_clean_scrub_stamp":"2024-05-03T18:44:45.380544+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:28:37.257537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":101,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.584399+0000","last_change":"2024-05-03T18:43:28.584846+0000","last_active":"2024-05-03T18:45:47.584399+0000","last_peered":"2024-05-03T18:45:47.584399+0000","last_clean":"2024-05-03T18:45:47.584399+0000","last_became_active":"2024-05-03T18:43:28.580087+0000","last_became_peered":"2024-05-03T18:43:28.580087+0000","last_unstale":"2024-05-03T18:45:47.584399+0000","last_undegraded":"2024-05-03T18:45:47.584399+0000","last_fullsized":"2024-05-03T18:45:47.584399+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:40:08.599278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:40:08.599278+0000","last_clean_scrub_stamp":"2024-05-03T18:40:08.599278+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:14:31.836127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.956184+0000","last_change":"2024-05-03T18:44:49.438545+0000","last_active":"2024-05-03T18:45:47.956184+0000","last_peered":"2024-05-03T18:45:47.956184+0000","last_clean":"2024-05-03T18:45:47.956184+0000","last_became_active":"2024-05-03T18:44:49.438178+0000","last_became_peered":"2024-05-03T18:44:49.438178+0000","last_unstale":"2024-05-03T18:45:47.956184+0000","last_undegraded":"2024-05-03T18:45:47.956184+0000","last_fullsized":"2024-05-03T18:45:47.956184+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:55:10.320262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"78'3","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.584446+0000","last_change":"2024-05-03T18:44:46.409567+0000","last_active":"2024-05-03T18:45:47.584446+0000","last_peered":"2024-05-03T18:45:47.584446+0000","last_clean":"2024-05-03T18:45:47.584446+0000","last_became_active":"2024-05-03T18:44:46.409078+0000","last_became_peered":"2024-05-03T18:44:46.409078+0000","last_unstale":"2024-05-03T18:45:47.584446+0000","last_undegraded":"2024-05-03T18:45:47.584446+0000","last_fullsized":"2024-05-03T18:45:47.584446+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_clean_scrub_stamp":"2024-05-03T18:44:45.380544+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:51:45.865890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"78'1","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.585437+0000","last_change":"2024-05-03T18:44:46.645061+0000","last_active":"2024-05-03T18:45:47.585437+0000","last_peered":"2024-05-03T18:45:47.585437+0000","last_clean":"2024-05-03T18:45:47.585437+0000","last_became_active":"2024-05-03T18:44:46.644631+0000","last_became_peered":"2024-05-03T18:44:46.644631+0000","last_unstale":"2024-05-03T18:45:47.585437+0000","last_undegraded":"2024-05-03T18:45:47.585437+0000","last_fullsized":"2024-05-03T18:45:47.585437+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_clean_scrub_stamp":"2024-05-03T18:44:45.380544+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:19:37.285434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.956494+0000","last_change":"2024-05-03T18:44:49.435674+0000","last_active":"2024-05-03T18:45:47.956494+0000","last_peered":"2024-05-03T18:45:47.956494+0000","last_clean":"2024-05-03T18:45:47.956494+0000","last_became_active":"2024-05-03T18:44:49.435333+0000","last_became_peered":"2024-05-03T18:44:49.435333+0000","last_unstale":"2024-05-03T18:45:47.956494+0000","last_undegraded":"2024-05-03T18:45:47.956494+0000","last_fullsized":"2024-05-03T18:45:47.956494+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:05:20.283773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"78'10","reported_seq":35,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.584714+0000","last_change":"2024-05-03T18:44:46.402696+0000","last_active":"2024-05-03T18:45:47.584714+0000","last_peered":"2024-05-03T18:45:47.584714+0000","last_clean":"2024-05-03T18:45:47.584714+0000","last_became_active":"2024-05-03T18:44:46.402288+0000","last_became_peered":"2024-05-03T18:44:46.402288+0000","last_unstale":"2024-05-03T18:45:47.584714+0000","last_undegraded":"2024-05-03T18:45:47.584714+0000","last_fullsized":"2024-05-03T18:45:47.584714+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_clean_scrub_stamp":"2024-05-03T18:44:45.380544+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:45:32.200586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.582839+0000","last_change":"2024-05-03T18:44:49.422754+0000","last_active":"2024-05-03T18:45:47.582839+0000","last_peered":"2024-05-03T18:45:47.582839+0000","last_clean":"2024-05-03T18:45:47.582839+0000","last_became_active":"2024-05-03T18:44:49.422385+0000","last_became_peered":"2024-05-03T18:44:49.422385+0000","last_unstale":"2024-05-03T18:45:47.582839+0000","last_undegraded":"2024-05-03T18:45:47.582839+0000","last_fullsized":"2024-05-03T18:45:47.582839+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T22:07:35.065842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"78'4","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.635326+0000","last_change":"2024-05-03T18:44:47.135851+0000","last_active":"2024-05-03T18:45:47.635326+0000","last_peered":"2024-05-03T18:45:47.635326+0000","last_clean":"2024-05-03T18:45:47.635326+0000","last_became_active":"2024-05-03T18:44:47.135480+0000","last_became_peered":"2024-05-03T18:44:47.135480+0000","last_unstale":"2024-05-03T18:45:47.635326+0000","last_undegraded":"2024-05-03T18:45:47.635326+0000","last_fullsized":"2024-05-03T18:45:47.635326+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_clean_scrub_stamp":"2024-05-03T18:44:45.380544+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:50:55.582018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.586246+0000","last_change":"2024-05-03T18:44:49.433123+0000","last_active":"2024-05-03T18:45:47.586246+0000","last_peered":"2024-05-03T18:45:47.586246+0000","last_clean":"2024-05-03T18:45:47.586246+0000","last_became_active":"2024-05-03T18:44:49.432781+0000","last_became_peered":"2024-05-03T18:44:49.432781+0000","last_unstale":"2024-05-03T18:45:47.586246+0000","last_undegraded":"2024-05-03T18:45:47.586246+0000","last_fullsized":"2024-05-03T18:45:47.586246+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:30:49.781052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"78'2","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.582904+0000","last_change":"2024-05-03T18:44:46.645267+0000","last_active":"2024-05-03T18:45:47.582904+0000","last_peered":"2024-05-03T18:45:47.582904+0000","last_clean":"2024-05-03T18:45:47.582904+0000","last_became_active":"2024-05-03T18:44:46.644804+0000","last_became_peered":"2024-05-03T18:44:46.644804+0000","last_unstale":"2024-05-03T18:45:47.582904+0000","last_undegraded":"2024-05-03T18:45:47.582904+0000","last_fullsized":"2024-05-03T18:45:47.582904+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_clean_scrub_stamp":"2024-05-03T18:44:45.380544+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:10:59.262218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.585467+0000","last_change":"2024-05-03T18:44:49.439352+0000","last_active":"2024-05-03T18:45:47.585467+0000","last_peered":"2024-05-03T18:45:47.585467+0000","last_clean":"2024-05-03T18:45:47.585467+0000","last_became_active":"2024-05-03T18:44:49.438981+0000","last_became_peered":"2024-05-03T18:44:49.438981+0000","last_unstale":"2024-05-03T18:45:47.585467+0000","last_undegraded":"2024-05-03T18:45:47.585467+0000","last_fullsized":"2024-05-03T18:45:47.585467+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T06:04:25.566153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"78'3","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.582432+0000","last_change":"2024-05-03T18:44:46.402305+0000","last_active":"2024-05-03T18:45:47.582432+0000","last_peered":"2024-05-03T18:45:47.582432+0000","last_clean":"2024-05-03T18:45:47.582432+0000","last_became_active":"2024-05-03T18:44:46.401633+0000","last_became_peered":"2024-05-03T18:44:46.401633+0000","last_unstale":"2024-05-03T18:45:47.582432+0000","last_undegraded":"2024-05-03T18:45:47.582432+0000","last_fullsized":"2024-05-03T18:45:47.582432+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_clean_scrub_stamp":"2024-05-03T18:44:45.380544+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:55:06.025508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.584759+0000","last_change":"2024-05-03T18:44:49.436728+0000","last_active":"2024-05-03T18:45:47.584759+0000","last_peered":"2024-05-03T18:45:47.584759+0000","last_clean":"2024-05-03T18:45:47.584759+0000","last_became_active":"2024-05-03T18:44:49.436356+0000","last_became_peered":"2024-05-03T18:44:49.436356+0000","last_unstale":"2024-05-03T18:45:47.584759+0000","last_undegraded":"2024-05-03T18:45:47.584759+0000","last_fullsized":"2024-05-03T18:45:47.584759+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:31:34.827237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"78'1","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.588472+0000","last_change":"2024-05-03T18:44:46.404949+0000","last_active":"2024-05-03T18:45:47.588472+0000","last_peered":"2024-05-03T18:45:47.588472+0000","last_clean":"2024-05-03T18:45:47.588472+0000","last_became_active":"2024-05-03T18:44:46.404516+0000","last_became_peered":"2024-05-03T18:44:46.404516+0000","last_unstale":"2024-05-03T18:45:47.588472+0000","last_undegraded":"2024-05-03T18:45:47.588472+0000","last_fullsized":"2024-05-03T18:45:47.588472+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_clean_scrub_stamp":"2024-05-03T18:44:45.380544+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:02:16.070763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.582603+0000","last_change":"2024-05-03T18:44:49.424090+0000","last_active":"2024-05-03T18:45:47.582603+0000","last_peered":"2024-05-03T18:45:47.582603+0000","last_clean":"2024-05-03T18:45:47.582603+0000","last_became_active":"2024-05-03T18:44:49.423110+0000","last_became_peered":"2024-05-03T18:44:49.423110+0000","last_unstale":"2024-05-03T18:45:47.582603+0000","last_undegraded":"2024-05-03T18:45:47.582603+0000","last_fullsized":"2024-05-03T18:45:47.582603+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:27:19.600924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"78'4","reported_seq":42,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.639194+0000","last_change":"2024-05-03T18:45:47.639194+0000","last_active":"2024-05-03T18:45:47.639194+0000","last_peered":"2024-05-03T18:45:47.639194+0000","last_clean":"2024-05-03T18:45:47.639194+0000","last_became_active":"2024-05-03T18:45:47.636067+0000","last_became_peered":"2024-05-03T18:45:47.636067+0000","last_unstale":"2024-05-03T18:45:47.639194+0000","last_undegraded":"2024-05-03T18:45:47.639194+0000","last_fullsized":"2024-05-03T18:45:47.639194+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_clean_scrub_stamp":"2024-05-03T18:44:45.380544+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T02:58:31.576713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.584592+0000","last_change":"2024-05-03T18:44:49.432498+0000","last_active":"2024-05-03T18:45:47.584592+0000","last_peered":"2024-05-03T18:45:47.584592+0000","last_clean":"2024-05-03T18:45:47.584592+0000","last_became_active":"2024-05-03T18:44:49.432152+0000","last_became_peered":"2024-05-03T18:44:49.432152+0000","last_unstale":"2024-05-03T18:45:47.584592+0000","last_undegraded":"2024-05-03T18:45:47.584592+0000","last_fullsized":"2024-05-03T18:45:47.584592+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:07:35.058213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"78'3","reported_seq":30,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.588551+0000","last_change":"2024-05-03T18:44:47.135973+0000","last_active":"2024-05-03T18:45:47.588551+0000","last_peered":"2024-05-03T18:45:47.588551+0000","last_clean":"2024-05-03T18:45:47.588551+0000","last_became_active":"2024-05-03T18:44:47.135548+0000","last_became_peered":"2024-05-03T18:44:47.135548+0000","last_unstale":"2024-05-03T18:45:47.588551+0000","last_undegraded":"2024-05-03T18:45:47.588551+0000","last_fullsized":"2024-05-03T18:45:47.588551+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_clean_scrub_stamp":"2024-05-03T18:44:45.380544+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:06:42.858462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.585036+0000","last_change":"2024-05-03T18:44:49.432722+0000","last_active":"2024-05-03T18:45:47.585036+0000","last_peered":"2024-05-03T18:45:47.585036+0000","last_clean":"2024-05-03T18:45:47.585036+0000","last_became_active":"2024-05-03T18:44:49.432393+0000","last_became_peered":"2024-05-03T18:44:49.432393+0000","last_unstale":"2024-05-03T18:45:47.585036+0000","last_undegraded":"2024-05-03T18:45:47.585036+0000","last_fullsized":"2024-05-03T18:45:47.585036+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:44:20.551442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"78'1","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.584489+0000","last_change":"2024-05-03T18:44:46.402489+0000","last_active":"2024-05-03T18:45:47.584489+0000","last_peered":"2024-05-03T18:45:47.584489+0000","last_clean":"2024-05-03T18:45:47.584489+0000","last_became_active":"2024-05-03T18:44:46.401992+0000","last_became_peered":"2024-05-03T18:44:46.401992+0000","last_unstale":"2024-05-03T18:45:47.584489+0000","last_undegraded":"2024-05-03T18:45:47.584489+0000","last_fullsized":"2024-05-03T18:45:47.584489+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_clean_scrub_stamp":"2024-05-03T18:44:45.380544+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:09:10.600254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"78'1","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.592504+0000","last_change":"2024-05-03T18:45:47.592504+0000","last_active":"2024-05-03T18:45:47.592504+0000","last_peered":"2024-05-03T18:45:47.592504+0000","last_clean":"2024-05-03T18:45:47.592504+0000","last_became_active":"2024-05-03T18:45:47.589068+0000","last_became_peered":"2024-05-03T18:45:47.589068+0000","last_unstale":"2024-05-03T18:45:47.592504+0000","last_undegraded":"2024-05-03T18:45:47.592504+0000","last_fullsized":"2024-05-03T18:45:47.592504+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:52:18.605331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,8],"acting":[7,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"78'2","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.956448+0000","last_change":"2024-05-03T18:44:46.404414+0000","last_active":"2024-05-03T18:45:47.956448+0000","last_peered":"2024-05-03T18:45:47.956448+0000","last_clean":"2024-05-03T18:45:47.956448+0000","last_became_active":"2024-05-03T18:44:46.404136+0000","last_became_peered":"2024-05-03T18:44:46.404136+0000","last_unstale":"2024-05-03T18:45:47.956448+0000","last_undegraded":"2024-05-03T18:45:47.956448+0000","last_fullsized":"2024-05-03T18:45:47.956448+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:45.380544+0000","last_clean_scrub_stamp":"2024-05-03T18:44:45.380544+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:24:58.125984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.584925+0000","last_change":"2024-05-03T18:44:49.429013+0000","last_active":"2024-05-03T18:45:47.584925+0000","last_peered":"2024-05-03T18:45:47.584925+0000","last_clean":"2024-05-03T18:45:47.584925+0000","last_became_active":"2024-05-03T18:44:49.428380+0000","last_became_peered":"2024-05-03T18:44:49.428380+0000","last_unstale":"2024-05-03T18:45:47.584925+0000","last_undegraded":"2024-05-03T18:45:47.584925+0000","last_fullsized":"2024-05-03T18:45:47.584925+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T04:13:00.073384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.584869+0000","last_change":"2024-05-03T18:44:49.437186+0000","last_active":"2024-05-03T18:45:47.584869+0000","last_peered":"2024-05-03T18:45:47.584869+0000","last_clean":"2024-05-03T18:45:47.584869+0000","last_became_active":"2024-05-03T18:44:49.435192+0000","last_became_peered":"2024-05-03T18:44:49.435192+0000","last_unstale":"2024-05-03T18:45:47.584869+0000","last_undegraded":"2024-05-03T18:45:47.584869+0000","last_fullsized":"2024-05-03T18:45:47.584869+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:16:25.707746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.589270+0000","last_change":"2024-05-03T18:44:49.441553+0000","last_active":"2024-05-03T18:45:47.589270+0000","last_peered":"2024-05-03T18:45:47.589270+0000","last_clean":"2024-05-03T18:45:47.589270+0000","last_became_active":"2024-05-03T18:44:49.441155+0000","last_became_peered":"2024-05-03T18:44:49.441155+0000","last_unstale":"2024-05-03T18:45:47.589270+0000","last_undegraded":"2024-05-03T18:45:47.589270+0000","last_fullsized":"2024-05-03T18:45:47.589270+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T00:53:53.888748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.588350+0000","last_change":"2024-05-03T18:44:49.436641+0000","last_active":"2024-05-03T18:45:47.588350+0000","last_peered":"2024-05-03T18:45:47.588350+0000","last_clean":"2024-05-03T18:45:47.588350+0000","last_became_active":"2024-05-03T18:44:49.436232+0000","last_became_peered":"2024-05-03T18:44:49.436232+0000","last_unstale":"2024-05-03T18:45:47.588350+0000","last_undegraded":"2024-05-03T18:45:47.588350+0000","last_fullsized":"2024-05-03T18:45:47.588350+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T21:06:55.773088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.586032+0000","last_change":"2024-05-03T18:44:49.434394+0000","last_active":"2024-05-03T18:45:47.586032+0000","last_peered":"2024-05-03T18:45:47.586032+0000","last_clean":"2024-05-03T18:45:47.586032+0000","last_became_active":"2024-05-03T18:44:49.433995+0000","last_became_peered":"2024-05-03T18:44:49.433995+0000","last_unstale":"2024-05-03T18:45:47.586032+0000","last_undegraded":"2024-05-03T18:45:47.586032+0000","last_fullsized":"2024-05-03T18:45:47.586032+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:25:20.478426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.956595+0000","last_change":"2024-05-03T18:44:49.439218+0000","last_active":"2024-05-03T18:45:47.956595+0000","last_peered":"2024-05-03T18:45:47.956595+0000","last_clean":"2024-05-03T18:45:47.956595+0000","last_became_active":"2024-05-03T18:44:49.438875+0000","last_became_peered":"2024-05-03T18:44:49.438875+0000","last_unstale":"2024-05-03T18:45:47.956595+0000","last_undegraded":"2024-05-03T18:45:47.956595+0000","last_fullsized":"2024-05-03T18:45:47.956595+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:08:55.899829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.584927+0000","last_change":"2024-05-03T18:44:49.433079+0000","last_active":"2024-05-03T18:45:47.584927+0000","last_peered":"2024-05-03T18:45:47.584927+0000","last_clean":"2024-05-03T18:45:47.584927+0000","last_became_active":"2024-05-03T18:44:49.432740+0000","last_became_peered":"2024-05-03T18:44:49.432740+0000","last_unstale":"2024-05-03T18:45:47.584927+0000","last_undegraded":"2024-05-03T18:45:47.584927+0000","last_fullsized":"2024-05-03T18:45:47.584927+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T19:31:36.317667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.589311+0000","last_change":"2024-05-03T18:44:49.435321+0000","last_active":"2024-05-03T18:45:47.589311+0000","last_peered":"2024-05-03T18:45:47.589311+0000","last_clean":"2024-05-03T18:45:47.589311+0000","last_became_active":"2024-05-03T18:44:49.434935+0000","last_became_peered":"2024-05-03T18:44:49.434935+0000","last_unstale":"2024-05-03T18:45:47.589311+0000","last_undegraded":"2024-05-03T18:45:47.589311+0000","last_fullsized":"2024-05-03T18:45:47.589311+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:14:32.406164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.582800+0000","last_change":"2024-05-03T18:44:49.438202+0000","last_active":"2024-05-03T18:45:47.582800+0000","last_peered":"2024-05-03T18:45:47.582800+0000","last_clean":"2024-05-03T18:45:47.582800+0000","last_became_active":"2024-05-03T18:44:49.437822+0000","last_became_peered":"2024-05-03T18:44:49.437822+0000","last_unstale":"2024-05-03T18:45:47.582800+0000","last_undegraded":"2024-05-03T18:45:47.582800+0000","last_fullsized":"2024-05-03T18:45:47.582800+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T01:12:54.417022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.583991+0000","last_change":"2024-05-03T18:44:49.434992+0000","last_active":"2024-05-03T18:45:47.583991+0000","last_peered":"2024-05-03T18:45:47.583991+0000","last_clean":"2024-05-03T18:45:47.583991+0000","last_became_active":"2024-05-03T18:44:49.434668+0000","last_became_peered":"2024-05-03T18:44:49.434668+0000","last_unstale":"2024-05-03T18:45:47.583991+0000","last_undegraded":"2024-05-03T18:45:47.583991+0000","last_fullsized":"2024-05-03T18:45:47.583991+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-04T23:20:48.527470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.582372+0000","last_change":"2024-05-03T18:44:49.432690+0000","last_active":"2024-05-03T18:45:47.582372+0000","last_peered":"2024-05-03T18:45:47.582372+0000","last_clean":"2024-05-03T18:45:47.582372+0000","last_became_active":"2024-05-03T18:44:49.432389+0000","last_became_peered":"2024-05-03T18:44:49.432389+0000","last_unstale":"2024-05-03T18:45:47.582372+0000","last_undegraded":"2024-05-03T18:45:47.582372+0000","last_fullsized":"2024-05-03T18:45:47.582372+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T05:51:27.326901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.583214+0000","last_change":"2024-05-03T18:44:49.434205+0000","last_active":"2024-05-03T18:45:47.583214+0000","last_peered":"2024-05-03T18:45:47.583214+0000","last_clean":"2024-05-03T18:45:47.583214+0000","last_became_active":"2024-05-03T18:44:49.433619+0000","last_became_peered":"2024-05-03T18:44:49.433619+0000","last_unstale":"2024-05-03T18:45:47.583214+0000","last_undegraded":"2024-05-03T18:45:47.583214+0000","last_fullsized":"2024-05-03T18:45:47.583214+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:36:15.933276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.956486+0000","last_change":"2024-05-03T18:44:49.425276+0000","last_active":"2024-05-03T18:45:47.956486+0000","last_peered":"2024-05-03T18:45:47.956486+0000","last_clean":"2024-05-03T18:45:47.956486+0000","last_became_active":"2024-05-03T18:44:49.424916+0000","last_became_peered":"2024-05-03T18:44:49.424916+0000","last_unstale":"2024-05-03T18:45:47.956486+0000","last_undegraded":"2024-05-03T18:45:47.956486+0000","last_fullsized":"2024-05-03T18:45:47.956486+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:09:03.512860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-05-03T18:45:47.585105+0000","last_change":"2024-05-03T18:44:49.433335+0000","last_active":"2024-05-03T18:45:47.585105+0000","last_peered":"2024-05-03T18:45:47.585105+0000","last_clean":"2024-05-03T18:45:47.585105+0000","last_became_active":"2024-05-03T18:44:49.432991+0000","last_became_peered":"2024-05-03T18:44:49.432991+0000","last_unstale":"2024-05-03T18:45:47.585105+0000","last_undegraded":"2024-05-03T18:45:47.585105+0000","last_fullsized":"2024-05-03T18:45:47.585105+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-03T18:44:48.401212+0000","last_clean_scrub_stamp":"2024-05-03T18:44:48.401212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-05T03:31:25.182590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":55834,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":587,"num_read_kb":352,"num_write":159,"num_write_kb":101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":167502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":136,"ondisk_log_size":136,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710757,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27876,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717276,"statfs":{"total":95995035648,"available":95966490624,"internally_reserved":0,"allocated":872448,"data_stored":425312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906986,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28548,"kb_used_data":1524,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716604,"statfs":{"total":95995035648,"available":95965802496,"internally_reserved":0,"allocated":1560576,"data_stored":1068626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070511,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":888832,"data_stored":425548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266740,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27892,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717260,"statfs":{"total":95995035648,"available":95966474240,"internally_reserved":0,"allocated":892928,"data_stored":425585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462970,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40952,"kb_used_data":1448,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704200,"statfs":{"total":95995035648,"available":95953100800,"internally_reserved":0,"allocated":1482752,"data_stored":1017398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626493,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27880,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717272,"statfs":{"total":95995035648,"available":95966486528,"internally_reserved":0,"allocated":876544,"data_stored":424937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574935,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":929792,"data_stored":476563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738459,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36216,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708936,"statfs":{"total":95995035648,"available":95957950464,"internally_reserved":0,"allocated":897024,"data_stored":426208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411410,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":954368,"data_stored":478304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182477,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36208,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708944,"statfs":{"total":95995035648,"available":95957958656,"internally_reserved":0,"allocated":892928,"data_stored":425248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986248,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":880640,"data_stored":425374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822723,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36796,"kb_used_data":1456,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708356,"statfs":{"total":95995035648,"available":95957356544,"internally_reserved":0,"allocated":1490944,"data_stored":1016965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":51833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":53574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":53411,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-03T18:46:43.383 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-05-03T18:46:43.383 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-05-03T18:46:43.383 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-05-03T18:46:43.384 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid f500455e-097b-11ef-bc97-c7b262605968 -- ceph health --format=json 2024-05-03T18:46:43.497 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:46:43 smithi045 ceph-mon[33299]: pgmap v338: 97 pgs: 97 active+clean; 631 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:46:43.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:46:43 smithi045 ceph-mon[33299]: from='client.14934 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-03T18:46:43.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:46:43 smithi113 ceph-mon[33540]: pgmap v338: 97 pgs: 97 active+clean; 631 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:46:43.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:46:43 smithi113 ceph-mon[33540]: from='client.14934 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-03T18:46:43.640 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:46:45.037 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:46:44 smithi032 ceph-mon[29763]: pgmap v339: 97 pgs: 97 active+clean; 631 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:46:45.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:46:44 smithi113 ceph-mon[33540]: pgmap v339: 97 pgs: 97 active+clean; 631 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:46:45.247 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:46:44 smithi045 ceph-mon[33299]: pgmap v339: 97 pgs: 97 active+clean; 631 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:46:45.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:45.252+0000 7f6ca7bb0700 1 -- 172.21.15.32:0/1903229451 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ca0106320 msgr2=0x7f6ca0106700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:45.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:45.252+0000 7f6ca7bb0700 1 --2- 172.21.15.32:0/1903229451 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ca0106320 0x7f6ca0106700 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f6c9c0099c0 tx=0x7f6c9c009cd0 comp rx=0 tx=0).stop 2024-05-03T18:46:45.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:45.252+0000 7f6ca7bb0700 1 -- 172.21.15.32:0/1903229451 shutdown_connections 2024-05-03T18:46:45.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:45.252+0000 7f6ca7bb0700 1 --2- 172.21.15.32:0/1903229451 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6ca0108b50 0x7f6ca010f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:45.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:45.252+0000 7f6ca7bb0700 1 --2- 172.21.15.32:0/1903229451 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ca0106320 0x7f6ca0106700 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:45.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:45.252+0000 7f6ca7bb0700 1 --2- 172.21.15.32:0/1903229451 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6ca0108190 0x7f6ca0108610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:45.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:45.252+0000 7f6ca7bb0700 1 -- 172.21.15.32:0/1903229451 >> 172.21.15.32:0/1903229451 conn(0x7f6ca00776c0 msgr2=0x7f6ca0077ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:46:45.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:45.253+0000 7f6ca7bb0700 1 -- 172.21.15.32:0/1903229451 shutdown_connections 2024-05-03T18:46:45.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:45.253+0000 7f6ca7bb0700 1 -- 172.21.15.32:0/1903229451 wait complete. 2024-05-03T18:46:45.262 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:45.253+0000 7f6ca7bb0700 1 Processor -- start 2024-05-03T18:46:45.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:45.253+0000 7f6ca7bb0700 1 -- start start 2024-05-03T18:46:45.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:45.253+0000 7f6ca7bb0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ca0106320 0x7f6ca019f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:45.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:45.253+0000 7f6ca7bb0700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6ca0108190 0x7f6ca019fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:45.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:45.253+0000 7f6ca7bb0700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6ca0108b50 0x7f6ca01999b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:45.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:45.253+0000 7f6ca7bb0700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f6ca01136f0 con 0x7f6ca0106320 2024-05-03T18:46:45.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:45.253+0000 7f6ca7bb0700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6ca0113570 con 0x7f6ca0108190 2024-05-03T18:46:45.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:45.253+0000 7f6ca7bb0700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f6ca0113870 con 0x7f6ca0108b50 2024-05-03T18:46:45.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:45.254+0000 7f6ca594c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ca0106320 0x7f6ca019f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:45.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:45.254+0000 7f6ca594c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ca0106320 0x7f6ca019f7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54174/0 (socket says 172.21.15.32:54174) 2024-05-03T18:46:45.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:45.254+0000 7f6ca594c700 1 -- 172.21.15.32:0/2148126341 learned_addr learned my addr 172.21.15.32:0/2148126341 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:46:45.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:45.254+0000 7f6ca514b700 1 --2- 172.21.15.32:0/2148126341 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6ca0108190 0x7f6ca019fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:45.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:45.254+0000 7f6ca614d700 1 --2- 172.21.15.32:0/2148126341 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6ca0108b50 0x7f6ca01999b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:45.263 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:45.254+0000 7f6ca594c700 1 -- 172.21.15.32:0/2148126341 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6ca0108b50 msgr2=0x7f6ca01999b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:45.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:45.254+0000 7f6ca594c700 1 --2- 172.21.15.32:0/2148126341 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6ca0108b50 0x7f6ca01999b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:45.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:45.254+0000 7f6ca594c700 1 -- 172.21.15.32:0/2148126341 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6ca0108190 msgr2=0x7f6ca019fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:45.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:45.254+0000 7f6ca594c700 1 --2- 172.21.15.32:0/2148126341 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6ca0108190 0x7f6ca019fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:45.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:45.254+0000 7f6ca594c700 1 -- 172.21.15.32:0/2148126341 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ca019a110 con 0x7f6ca0106320 2024-05-03T18:46:45.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:45.254+0000 7f6ca514b700 1 --2- 172.21.15.32:0/2148126341 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6ca0108190 0x7f6ca019fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T18:46:45.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:45.254+0000 7f6ca594c700 1 --2- 172.21.15.32:0/2148126341 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ca0106320 0x7f6ca019f7f0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f6c9c006010 tx=0x7f6c9c004690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:46:45.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:45.254+0000 7f6c96ffd700 1 -- 172.21.15.32:0/2148126341 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c9c026070 con 0x7f6ca0106320 2024-05-03T18:46:45.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:45.254+0000 7f6c96ffd700 1 -- 172.21.15.32:0/2148126341 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c9c00f460 con 0x7f6ca0106320 2024-05-03T18:46:45.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:45.254+0000 7f6ca7bb0700 1 -- 172.21.15.32:0/2148126341 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ca019a3a0 con 0x7f6ca0106320 2024-05-03T18:46:45.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:45.254+0000 7f6c96ffd700 1 -- 172.21.15.32:0/2148126341 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c9c009e60 con 0x7f6ca0106320 2024-05-03T18:46:45.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:45.254+0000 7f6ca7bb0700 1 -- 172.21.15.32:0/2148126341 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ca01a66c0 con 0x7f6ca0106320 2024-05-03T18:46:45.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:45.255+0000 7f6ca7bb0700 1 -- 172.21.15.32:0/2148126341 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ca006b7f0 con 0x7f6ca0106320 2024-05-03T18:46:45.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:45.256+0000 7f6c96ffd700 1 -- 172.21.15.32:0/2148126341 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f6c9c029070 con 0x7f6ca0106320 2024-05-03T18:46:45.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:45.257+0000 7f6c96ffd700 1 --2- 172.21.15.32:0/2148126341 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f6c8c0b2910 0x7f6c8c0b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:45.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:45.257+0000 7f6ca514b700 1 --2- 172.21.15.32:0/2148126341 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f6c8c0b2910 0x7f6c8c0b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:45.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:45.257+0000 7f6c96ffd700 1 -- 172.21.15.32:0/2148126341 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8269+0+0 (secure 0 0 0) 0x7f6c9c0e2e80 con 0x7f6ca0106320 2024-05-03T18:46:45.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:45.258+0000 7f6ca514b700 1 --2- 172.21.15.32:0/2148126341 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f6c8c0b2910 0x7f6c8c0b4dd0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f6ca0077c50 tx=0x7f6c90009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:46:45.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:45.261+0000 7f6c96ffd700 1 -- 172.21.15.32:0/2148126341 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f6c9c030080 con 0x7f6ca0106320 2024-05-03T18:46:45.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:45.526+0000 7f6ca7bb0700 1 -- 172.21.15.32:0/2148126341 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f6ca019aee0 con 0x7f6ca0106320 2024-05-03T18:46:45.527 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:45.526+0000 7f6c96ffd700 1 -- 172.21.15.32:0/2148126341 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f6c9c013070 con 0x7f6ca0106320 2024-05-03T18:46:45.527 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-03T18:46:45.528 INFO:teuthology.orchestra.run.smithi032.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-05-03T18:46:45.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:45.529+0000 7f6ca7bb0700 1 -- 172.21.15.32:0/2148126341 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f6c8c0b2910 msgr2=0x7f6c8c0b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:45.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:45.529+0000 7f6ca7bb0700 1 --2- 172.21.15.32:0/2148126341 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f6c8c0b2910 0x7f6c8c0b4dd0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f6ca0077c50 tx=0x7f6c90009260 comp rx=0 tx=0).stop 2024-05-03T18:46:45.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:45.529+0000 7f6ca7bb0700 1 -- 172.21.15.32:0/2148126341 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ca0106320 msgr2=0x7f6ca019f7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:45.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:45.529+0000 7f6ca7bb0700 1 --2- 172.21.15.32:0/2148126341 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ca0106320 0x7f6ca019f7f0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f6c9c006010 tx=0x7f6c9c004690 comp rx=0 tx=0).stop 2024-05-03T18:46:45.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:45.529+0000 7f6ca7bb0700 1 -- 172.21.15.32:0/2148126341 shutdown_connections 2024-05-03T18:46:45.530 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:45.529+0000 7f6ca7bb0700 1 --2- 172.21.15.32:0/2148126341 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f6c8c0b2910 0x7f6c8c0b4dd0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:45.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:45.529+0000 7f6ca7bb0700 1 --2- 172.21.15.32:0/2148126341 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6ca0108b50 0x7f6ca01999b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:45.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:45.529+0000 7f6ca7bb0700 1 --2- 172.21.15.32:0/2148126341 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ca0106320 0x7f6ca019f7f0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:45.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:45.529+0000 7f6ca7bb0700 1 --2- 172.21.15.32:0/2148126341 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6ca0108190 0x7f6ca019fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:45.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:45.529+0000 7f6ca7bb0700 1 -- 172.21.15.32:0/2148126341 >> 172.21.15.32:0/2148126341 conn(0x7f6ca00776c0 msgr2=0x7f6ca00ffe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:46:45.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:45.529+0000 7f6ca7bb0700 1 -- 172.21.15.32:0/2148126341 shutdown_connections 2024-05-03T18:46:45.531 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:45.529+0000 7f6ca7bb0700 1 -- 172.21.15.32:0/2148126341 wait complete. 2024-05-03T18:46:46.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:46:45 smithi113 ceph-mon[33540]: from='client.? 172.21.15.32:0/2148126341' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-03T18:46:46.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:46:45 smithi032 ceph-mon[29763]: from='client.? 172.21.15.32:0/2148126341' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-03T18:46:46.247 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:46:45 smithi045 ceph-mon[33299]: from='client.? 172.21.15.32:0/2148126341' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-03T18:46:46.263 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-05-03T18:46:46.264 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid f500455e-097b-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-03T18:46:46.514 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:46:46.890 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:46:46 smithi032 ceph-mon[29763]: pgmap v340: 97 pgs: 97 active+clean; 631 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:46:46.890 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:46:46 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2024-05-03T18:46:46.890 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:46:46 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 1]}]: dispatch 2024-05-03T18:46:46.890 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:46:46 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 11]}]: dispatch 2024-05-03T18:46:47.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:46:46 smithi113 ceph-mon[33540]: pgmap v340: 97 pgs: 97 active+clean; 631 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:46:47.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:46:46 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2024-05-03T18:46:47.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:46:46 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 1]}]: dispatch 2024-05-03T18:46:47.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:46:46 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 11]}]: dispatch 2024-05-03T18:46:47.248 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:46:46 smithi045 ceph-mon[33299]: pgmap v340: 97 pgs: 97 active+clean; 631 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:46:47.248 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:46:46 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2024-05-03T18:46:47.248 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:46:46 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 1]}]: dispatch 2024-05-03T18:46:47.248 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:46:46 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 11]}]: dispatch 2024-05-03T18:46:47.918 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:46:47 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-05-03T18:46:47.918 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:46:47 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 1]}]': finished 2024-05-03T18:46:47.918 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:46:47 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 11]}]': finished 2024-05-03T18:46:47.918 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:46:47 smithi032 ceph-mon[29763]: osdmap e81: 12 total, 12 up, 12 in 2024-05-03T18:46:48.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:46:47 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-05-03T18:46:48.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:46:47 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 1]}]': finished 2024-05-03T18:46:48.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:46:47 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 11]}]': finished 2024-05-03T18:46:48.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:46:47 smithi113 ceph-mon[33540]: osdmap e81: 12 total, 12 up, 12 in 2024-05-03T18:46:48.140 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:48.138+0000 7fae4961e700 1 -- 172.21.15.32:0/2619742366 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae44108850 msgr2=0x7fae441103d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:48.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:48.138+0000 7fae4961e700 1 --2- 172.21.15.32:0/2619742366 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae44108850 0x7fae441103d0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fae3c00b0d0 tx=0x7fae3c00b3e0 comp rx=0 tx=0).stop 2024-05-03T18:46:48.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:48.138+0000 7fae4961e700 1 -- 172.21.15.32:0/2619742366 shutdown_connections 2024-05-03T18:46:48.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:48.138+0000 7fae4961e700 1 --2- 172.21.15.32:0/2619742366 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae44108850 0x7fae441103d0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:48.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:48.138+0000 7fae4961e700 1 --2- 172.21.15.32:0/2619742366 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fae44103750 0x7fae44103b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:48.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:48.138+0000 7fae4961e700 1 --2- 172.21.15.32:0/2619742366 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fae44104100 0x7fae44108120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:48.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:48.138+0000 7fae4961e700 1 -- 172.21.15.32:0/2619742366 >> 172.21.15.32:0/2619742366 conn(0x7fae4407a600 msgr2=0x7fae4407aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:46:48.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:48.138+0000 7fae4961e700 1 -- 172.21.15.32:0/2619742366 shutdown_connections 2024-05-03T18:46:48.141 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:48.138+0000 7fae4961e700 1 -- 172.21.15.32:0/2619742366 wait complete. 2024-05-03T18:46:48.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:48.139+0000 7fae4961e700 1 Processor -- start 2024-05-03T18:46:48.143 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:48.139+0000 7fae4961e700 1 -- start start 2024-05-03T18:46:48.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:48.139+0000 7fae4961e700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fae44103750 0x7fae440757a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:48.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:48.139+0000 7fae4961e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae44104100 0x7fae44075ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:48.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:48.139+0000 7fae4961e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fae44108850 0x7fae44076220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:48.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:48.139+0000 7fae4961e700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fae44113640 con 0x7fae44104100 2024-05-03T18:46:48.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:48.139+0000 7fae4961e700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fae441134c0 con 0x7fae44108850 2024-05-03T18:46:48.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:48.139+0000 7fae4961e700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fae441137c0 con 0x7fae44103750 2024-05-03T18:46:48.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:48.139+0000 7fae427fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae44104100 0x7fae44075ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:48.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:48.139+0000 7fae427fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae44104100 0x7fae44075ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54196/0 (socket says 172.21.15.32:54196) 2024-05-03T18:46:48.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:48.139+0000 7fae427fc700 1 -- 172.21.15.32:0/785816147 learned_addr learned my addr 172.21.15.32:0/785816147 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:46:48.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:48.140+0000 7fae42ffd700 1 --2- 172.21.15.32:0/785816147 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fae44103750 0x7fae440757a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:48.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:48.140+0000 7fae437fe700 1 --2- 172.21.15.32:0/785816147 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fae44108850 0x7fae44076220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:48.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:48.140+0000 7fae427fc700 1 -- 172.21.15.32:0/785816147 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fae44103750 msgr2=0x7fae440757a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:48.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:48.140+0000 7fae427fc700 1 --2- 172.21.15.32:0/785816147 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fae44103750 0x7fae440757a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:48.144 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:48.140+0000 7fae427fc700 1 -- 172.21.15.32:0/785816147 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fae44108850 msgr2=0x7fae44076220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:48.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:48.140+0000 7fae427fc700 1 --2- 172.21.15.32:0/785816147 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fae44108850 0x7fae44076220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:48.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:48.140+0000 7fae427fc700 1 -- 172.21.15.32:0/785816147 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae441a8020 con 0x7fae44104100 2024-05-03T18:46:48.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:48.140+0000 7fae437fe700 1 --2- 172.21.15.32:0/785816147 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fae44108850 0x7fae44076220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:46:48.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:48.140+0000 7fae427fc700 1 --2- 172.21.15.32:0/785816147 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae44104100 0x7fae44075ce0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fae3800c970 tx=0x7fae3800cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:46:48.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:48.140+0000 7fae2bfff700 1 -- 172.21.15.32:0/785816147 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae38007aa0 con 0x7fae44104100 2024-05-03T18:46:48.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:48.140+0000 7fae2bfff700 1 -- 172.21.15.32:0/785816147 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae38004ce0 con 0x7fae44104100 2024-05-03T18:46:48.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:48.140+0000 7fae4961e700 1 -- 172.21.15.32:0/785816147 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae441a8310 con 0x7fae44104100 2024-05-03T18:46:48.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:48.140+0000 7fae2bfff700 1 -- 172.21.15.32:0/785816147 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae380055c0 con 0x7fae44104100 2024-05-03T18:46:48.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:48.140+0000 7fae4961e700 1 -- 172.21.15.32:0/785816147 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae441a8800 con 0x7fae44104100 2024-05-03T18:46:48.145 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:48.141+0000 7fae4961e700 1 -- 172.21.15.32:0/785816147 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae4406a190 con 0x7fae44104100 2024-05-03T18:46:48.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:48.142+0000 7fae2bfff700 1 -- 172.21.15.32:0/785816147 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fae38007c40 con 0x7fae44104100 2024-05-03T18:46:48.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:48.143+0000 7fae2bfff700 1 --2- 172.21.15.32:0/785816147 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fae2c0b2910 0x7fae2c0b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:48.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:48.143+0000 7fae42ffd700 1 --2- 172.21.15.32:0/785816147 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fae2c0b2910 0x7fae2c0b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:48.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:48.143+0000 7fae2bfff700 1 -- 172.21.15.32:0/785816147 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8356+0+0 (secure 0 0 0) 0x7fae380e2520 con 0x7fae44104100 2024-05-03T18:46:48.146 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:48.144+0000 7fae42ffd700 1 --2- 172.21.15.32:0/785816147 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fae2c0b2910 0x7fae2c0b4dd0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fae34005950 tx=0x7fae340058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:46:48.148 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:48.147+0000 7fae2bfff700 1 -- 172.21.15.32:0/785816147 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fae380aa120 con 0x7fae44104100 2024-05-03T18:46:48.248 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:46:47 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-05-03T18:46:48.248 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:46:47 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 1]}]': finished 2024-05-03T18:46:48.248 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:46:47 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 11]}]': finished 2024-05-03T18:46:48.248 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:46:47 smithi045 ceph-mon[33299]: osdmap e81: 12 total, 12 up, 12 in 2024-05-03T18:46:48.413 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:48.411+0000 7fae4961e700 1 -- 172.21.15.32:0/785816147 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fae4402e510 con 0x7fae44104100 2024-05-03T18:46:48.414 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:48.412+0000 7fae2bfff700 1 -- 172.21.15.32:0/785816147 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11862 (secure 0 0 0) 0x7fae380a9a20 con 0x7fae44104100 2024-05-03T18:46:48.415 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-03T18:46:48.415 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/535599929","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":535599929},{"type":"v1","addr":"172.21.15.32:6835","nonce":535599929}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/3101679868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":3101679868},{"type":"v1","addr":"172.21.15.32:6837","nonce":3101679868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14610,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/3838653921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":3838653921},{"type":"v1","addr":"172.21.15.32:6839","nonce":3838653921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14622,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6841/3019648436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6840","nonce":3019648436},{"type":"v1","addr":"172.21.15.32:6841","nonce":3019648436}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24380,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/1629869435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":1629869435},{"type":"v1","addr":"172.21.15.45:6833","nonce":1629869435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24398,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2997291166","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2997291166},{"type":"v1","addr":"172.21.15.45:6835","nonce":2997291166}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-03T18:44:51.155050+0000","modified":"2024-05-03T18:45:08.404530+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24478,"mds_1":24469,"mds_2":24463},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24425":{"gid":24425,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.45:6837/428055392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":428055392},{"type":"v1","addr":"172.21.15.45:6837","nonce":428055392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.113:6833/3323558791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":3323558791},{"type":"v1","addr":"172.21.15.113:6833","nonce":3323558791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"b","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.45:6839/970804040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":970804040},{"type":"v1","addr":"172.21.15.45:6839","nonce":970804040}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.113:6835/2588738276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2588738276},{"type":"v1","addr":"172.21.15.113:6835","nonce":2588738276}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.113:6837/3677284460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":3677284460},{"type":"v1","addr":"172.21.15.113:6837","nonce":3677284460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.113:6839/942050693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6838","nonce":942050693},{"type":"v1","addr":"172.21.15.113:6839","nonce":942050693}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24478,"qdb_cluster":[24469,24478,24463]},"id":1}]} 2024-05-03T18:46:48.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:48.414+0000 7fae4961e700 1 -- 172.21.15.32:0/785816147 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fae2c0b2910 msgr2=0x7fae2c0b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:48.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:48.414+0000 7fae4961e700 1 --2- 172.21.15.32:0/785816147 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fae2c0b2910 0x7fae2c0b4dd0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fae34005950 tx=0x7fae340058e0 comp rx=0 tx=0).stop 2024-05-03T18:46:48.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:48.414+0000 7fae4961e700 1 -- 172.21.15.32:0/785816147 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae44104100 msgr2=0x7fae44075ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:48.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:48.414+0000 7fae4961e700 1 --2- 172.21.15.32:0/785816147 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae44104100 0x7fae44075ce0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fae3800c970 tx=0x7fae3800cd30 comp rx=0 tx=0).stop 2024-05-03T18:46:48.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:48.414+0000 7fae4961e700 1 -- 172.21.15.32:0/785816147 shutdown_connections 2024-05-03T18:46:48.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:48.414+0000 7fae4961e700 1 --2- 172.21.15.32:0/785816147 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fae2c0b2910 0x7fae2c0b4dd0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:48.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:48.414+0000 7fae4961e700 1 --2- 172.21.15.32:0/785816147 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fae44108850 0x7fae44076220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:48.417 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:48.414+0000 7fae4961e700 1 --2- 172.21.15.32:0/785816147 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fae44103750 0x7fae440757a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:48.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:48.414+0000 7fae4961e700 1 --2- 172.21.15.32:0/785816147 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae44104100 0x7fae44075ce0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:48.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:48.414+0000 7fae4961e700 1 -- 172.21.15.32:0/785816147 >> 172.21.15.32:0/785816147 conn(0x7fae4407a600 msgr2=0x7fae44101540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:46:48.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:48.415+0000 7fae4961e700 1 -- 172.21.15.32:0/785816147 shutdown_connections 2024-05-03T18:46:48.418 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:48.415+0000 7fae4961e700 1 -- 172.21.15.32:0/785816147 wait complete. 2024-05-03T18:46:48.418 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 25 2024-05-03T18:46:49.041 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:46:48 smithi032 ceph-mon[29763]: osdmap e82: 12 total, 12 up, 12 in 2024-05-03T18:46:49.041 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:46:48 smithi032 ceph-mon[29763]: pgmap v343: 97 pgs: 97 active+clean; 631 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-03T18:46:49.042 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:46:48 smithi032 ceph-mon[29763]: from='client.? 172.21.15.32:0/785816147' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:46:49.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:46:48 smithi113 ceph-mon[33540]: osdmap e82: 12 total, 12 up, 12 in 2024-05-03T18:46:49.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:46:48 smithi113 ceph-mon[33540]: pgmap v343: 97 pgs: 97 active+clean; 631 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-03T18:46:49.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:46:48 smithi113 ceph-mon[33540]: from='client.? 172.21.15.32:0/785816147' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:46:49.071 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid f500455e-097b-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-03T18:46:49.248 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:46:48 smithi045 ceph-mon[33299]: osdmap e82: 12 total, 12 up, 12 in 2024-05-03T18:46:49.248 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:46:48 smithi045 ceph-mon[33299]: pgmap v343: 97 pgs: 97 active+clean; 631 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-03T18:46:49.248 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:46:48 smithi045 ceph-mon[33299]: from='client.? 172.21.15.32:0/785816147' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:46:49.319 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:46:50.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:50.991+0000 7ff1e0e68700 1 -- 172.21.15.32:0/4198585810 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff1dc103390 msgr2=0x7ff1dc107000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:50.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:50.991+0000 7ff1e0e68700 1 --2- 172.21.15.32:0/4198585810 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff1dc103390 0x7ff1dc107000 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7ff1d00099c0 tx=0x7ff1d0009cd0 comp rx=0 tx=0).stop 2024-05-03T18:46:50.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:50.991+0000 7ff1e0e68700 1 -- 172.21.15.32:0/4198585810 shutdown_connections 2024-05-03T18:46:50.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:50.991+0000 7ff1e0e68700 1 --2- 172.21.15.32:0/4198585810 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1dc107730 0x7ff1dc10f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:50.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:50.991+0000 7ff1e0e68700 1 --2- 172.21.15.32:0/4198585810 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff1dc1029e0 0x7ff1dc102dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:50.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:50.991+0000 7ff1e0e68700 1 --2- 172.21.15.32:0/4198585810 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff1dc103390 0x7ff1dc107000 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:50.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:50.991+0000 7ff1e0e68700 1 -- 172.21.15.32:0/4198585810 >> 172.21.15.32:0/4198585810 conn(0x7ff1dc079350 msgr2=0x7ff1dc079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:46:50.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:50.991+0000 7ff1e0e68700 1 -- 172.21.15.32:0/4198585810 shutdown_connections 2024-05-03T18:46:50.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:50.991+0000 7ff1e0e68700 1 -- 172.21.15.32:0/4198585810 wait complete. 2024-05-03T18:46:50.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:50.992+0000 7ff1e0e68700 1 Processor -- start 2024-05-03T18:46:50.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:50.992+0000 7ff1e0e68700 1 -- start start 2024-05-03T18:46:50.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:50.992+0000 7ff1e0e68700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1dc1029e0 0x7ff1dc10fab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:50.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:50.992+0000 7ff1e0e68700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff1dc103390 0x7ff1dc10fff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:50.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:50.992+0000 7ff1e0e68700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff1dc107730 0x7ff1dc110530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:50.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:50.992+0000 7ff1e0e68700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff1dc113750 con 0x7ff1dc107730 2024-05-03T18:46:50.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:50.992+0000 7ff1e0e68700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff1dc1135d0 con 0x7ff1dc1029e0 2024-05-03T18:46:50.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:50.992+0000 7ff1e0e68700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7ff1dc1138d0 con 0x7ff1dc103390 2024-05-03T18:46:50.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:50.992+0000 7ff1dad9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff1dc107730 0x7ff1dc110530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:50.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:50.992+0000 7ff1dad9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff1dc107730 0x7ff1dc110530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54208/0 (socket says 172.21.15.32:54208) 2024-05-03T18:46:50.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:50.992+0000 7ff1dad9d700 1 -- 172.21.15.32:0/3647888463 learned_addr learned my addr 172.21.15.32:0/3647888463 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:46:50.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:50.993+0000 7ff1d9d9b700 1 --2- 172.21.15.32:0/3647888463 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff1dc103390 0x7ff1dc10fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:50.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:50.993+0000 7ff1da59c700 1 --2- 172.21.15.32:0/3647888463 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1dc1029e0 0x7ff1dc10fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:50.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:50.993+0000 7ff1dad9d700 1 -- 172.21.15.32:0/3647888463 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff1dc103390 msgr2=0x7ff1dc10fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:50.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:50.993+0000 7ff1dad9d700 1 --2- 172.21.15.32:0/3647888463 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff1dc103390 0x7ff1dc10fff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:50.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:50.993+0000 7ff1dad9d700 1 -- 172.21.15.32:0/3647888463 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1dc1029e0 msgr2=0x7ff1dc10fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:50.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:50.993+0000 7ff1dad9d700 1 --2- 172.21.15.32:0/3647888463 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1dc1029e0 0x7ff1dc10fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:50.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:50.993+0000 7ff1dad9d700 1 -- 172.21.15.32:0/3647888463 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1dc1ac650 con 0x7ff1dc107730 2024-05-03T18:46:50.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:50.993+0000 7ff1da59c700 1 --2- 172.21.15.32:0/3647888463 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1dc1029e0 0x7ff1dc10fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:46:50.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:50.993+0000 7ff1dad9d700 1 --2- 172.21.15.32:0/3647888463 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff1dc107730 0x7ff1dc110530 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7ff1d400b970 tx=0x7ff1d400bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:46:50.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:50.993+0000 7ff1cb7fe700 1 -- 172.21.15.32:0/3647888463 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1d4019070 con 0x7ff1dc107730 2024-05-03T18:46:50.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:50.993+0000 7ff1e0e68700 1 -- 172.21.15.32:0/3647888463 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1dc1ac8e0 con 0x7ff1dc107730 2024-05-03T18:46:50.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:50.993+0000 7ff1e0e68700 1 -- 172.21.15.32:0/3647888463 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1dc1acdf0 con 0x7ff1dc107730 2024-05-03T18:46:50.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:50.993+0000 7ff1cb7fe700 1 -- 172.21.15.32:0/3647888463 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1d400cde0 con 0x7ff1dc107730 2024-05-03T18:46:50.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:50.993+0000 7ff1cb7fe700 1 -- 172.21.15.32:0/3647888463 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1d401d3b0 con 0x7ff1dc107730 2024-05-03T18:46:50.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:50.994+0000 7ff1e0e68700 1 -- 172.21.15.32:0/3647888463 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1bc005330 con 0x7ff1dc107730 2024-05-03T18:46:50.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:50.996+0000 7ff1cb7fe700 1 -- 172.21.15.32:0/3647888463 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7ff1d401c070 con 0x7ff1dc107730 2024-05-03T18:46:50.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:50.996+0000 7ff1cb7fe700 1 --2- 172.21.15.32:0/3647888463 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7ff1c40b2990 0x7ff1c40b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:50.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:50.997+0000 7ff1da59c700 1 --2- 172.21.15.32:0/3647888463 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7ff1c40b2990 0x7ff1c40b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:50.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:50.997+0000 7ff1cb7fe700 1 -- 172.21.15.32:0/3647888463 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8356+0+0 (secure 0 0 0) 0x7ff1d40e1a10 con 0x7ff1dc107730 2024-05-03T18:46:50.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:50.997+0000 7ff1da59c700 1 --2- 172.21.15.32:0/3647888463 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7ff1c40b2990 0x7ff1c40b4e50 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc007900 tx=0x7ff1cc008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:46:51.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:51.000+0000 7ff1cb7fe700 1 -- 172.21.15.32:0/3647888463 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7ff1d40a9660 con 0x7ff1dc107730 2024-05-03T18:46:51.264 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:51.262+0000 7ff1e0e68700 1 -- 172.21.15.32:0/3647888463 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff1bc005620 con 0x7ff1dc107730 2024-05-03T18:46:51.265 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:46:51 smithi032 ceph-mon[29763]: pgmap v344: 97 pgs: 97 active+clean; 631 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2024-05-03T18:46:51.265 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:51.263+0000 7ff1cb7fe700 1 -- 172.21.15.32:0/3647888463 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11862 (secure 0 0 0) 0x7ff1d40a8f60 con 0x7ff1dc107730 2024-05-03T18:46:51.265 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-03T18:46:51.265 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/535599929","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":535599929},{"type":"v1","addr":"172.21.15.32:6835","nonce":535599929}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/3101679868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":3101679868},{"type":"v1","addr":"172.21.15.32:6837","nonce":3101679868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14610,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/3838653921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":3838653921},{"type":"v1","addr":"172.21.15.32:6839","nonce":3838653921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14622,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6841/3019648436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6840","nonce":3019648436},{"type":"v1","addr":"172.21.15.32:6841","nonce":3019648436}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24380,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/1629869435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":1629869435},{"type":"v1","addr":"172.21.15.45:6833","nonce":1629869435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24398,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2997291166","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2997291166},{"type":"v1","addr":"172.21.15.45:6835","nonce":2997291166}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-03T18:44:51.155050+0000","modified":"2024-05-03T18:45:08.404530+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24478,"mds_1":24469,"mds_2":24463},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24425":{"gid":24425,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.45:6837/428055392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":428055392},{"type":"v1","addr":"172.21.15.45:6837","nonce":428055392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.113:6833/3323558791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":3323558791},{"type":"v1","addr":"172.21.15.113:6833","nonce":3323558791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"b","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.45:6839/970804040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":970804040},{"type":"v1","addr":"172.21.15.45:6839","nonce":970804040}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.113:6835/2588738276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2588738276},{"type":"v1","addr":"172.21.15.113:6835","nonce":2588738276}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.113:6837/3677284460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":3677284460},{"type":"v1","addr":"172.21.15.113:6837","nonce":3677284460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.113:6839/942050693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6838","nonce":942050693},{"type":"v1","addr":"172.21.15.113:6839","nonce":942050693}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24478,"qdb_cluster":[24469,24478,24463]},"id":1}]} 2024-05-03T18:46:51.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:51.265+0000 7ff1e0e68700 1 -- 172.21.15.32:0/3647888463 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7ff1c40b2990 msgr2=0x7ff1c40b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:51.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:51.265+0000 7ff1e0e68700 1 --2- 172.21.15.32:0/3647888463 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7ff1c40b2990 0x7ff1c40b4e50 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc007900 tx=0x7ff1cc008040 comp rx=0 tx=0).stop 2024-05-03T18:46:51.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:51.265+0000 7ff1e0e68700 1 -- 172.21.15.32:0/3647888463 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff1dc107730 msgr2=0x7ff1dc110530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:51.267 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:51.265+0000 7ff1e0e68700 1 --2- 172.21.15.32:0/3647888463 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff1dc107730 0x7ff1dc110530 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7ff1d400b970 tx=0x7ff1d400bc80 comp rx=0 tx=0).stop 2024-05-03T18:46:51.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:51.265+0000 7ff1e0e68700 1 -- 172.21.15.32:0/3647888463 shutdown_connections 2024-05-03T18:46:51.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:51.265+0000 7ff1e0e68700 1 --2- 172.21.15.32:0/3647888463 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7ff1c40b2990 0x7ff1c40b4e50 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:51.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:51.265+0000 7ff1e0e68700 1 --2- 172.21.15.32:0/3647888463 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff1dc107730 0x7ff1dc110530 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:51.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:51.265+0000 7ff1e0e68700 1 --2- 172.21.15.32:0/3647888463 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1dc1029e0 0x7ff1dc10fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:51.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:51.265+0000 7ff1e0e68700 1 --2- 172.21.15.32:0/3647888463 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff1dc103390 0x7ff1dc10fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:51.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:51.265+0000 7ff1e0e68700 1 -- 172.21.15.32:0/3647888463 >> 172.21.15.32:0/3647888463 conn(0x7ff1dc079350 msgr2=0x7ff1dc100b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:46:51.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:51.265+0000 7ff1e0e68700 1 -- 172.21.15.32:0/3647888463 shutdown_connections 2024-05-03T18:46:51.268 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:51.265+0000 7ff1e0e68700 1 -- 172.21.15.32:0/3647888463 wait complete. 2024-05-03T18:46:51.268 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 25 2024-05-03T18:46:51.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:46:51 smithi113 ceph-mon[33540]: pgmap v344: 97 pgs: 97 active+clean; 631 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2024-05-03T18:46:51.497 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:46:51 smithi045 ceph-mon[33299]: pgmap v344: 97 pgs: 97 active+clean; 631 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2024-05-03T18:46:51.957 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid f500455e-097b-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-03T18:46:52.213 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:46:52.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:46:52 smithi032 ceph-mon[29763]: from='client.? 172.21.15.32:0/3647888463' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:46:52.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:46:52 smithi045 ceph-mon[33299]: from='client.? 172.21.15.32:0/3647888463' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:46:52.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:46:52 smithi113 ceph-mon[33540]: from='client.? 172.21.15.32:0/3647888463' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:46:53.199 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:46:53 smithi032 ceph-mon[29763]: pgmap v345: 97 pgs: 97 active+clean; 631 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 127 B/s wr, 5 op/s; 75 B/s, 1 keys/s, 0 objects/s recovering 2024-05-03T18:46:53.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:46:53 smithi045 ceph-mon[33299]: pgmap v345: 97 pgs: 97 active+clean; 631 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 127 B/s wr, 5 op/s; 75 B/s, 1 keys/s, 0 objects/s recovering 2024-05-03T18:46:53.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:46:53 smithi113 ceph-mon[33540]: pgmap v345: 97 pgs: 97 active+clean; 631 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 127 B/s wr, 5 op/s; 75 B/s, 1 keys/s, 0 objects/s recovering 2024-05-03T18:46:53.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:53.691+0000 7f72b375c700 1 -- 172.21.15.32:0/1577322762 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72ac102d50 msgr2=0x7f72ac10f240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:53.692 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:53.691+0000 7f72b375c700 1 --2- 172.21.15.32:0/1577322762 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72ac102d50 0x7f72ac10f240 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f72a400b0a0 tx=0x7f72a400b3b0 comp rx=0 tx=0).stop 2024-05-03T18:46:53.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:53.691+0000 7f72b375c700 1 -- 172.21.15.32:0/1577322762 shutdown_connections 2024-05-03T18:46:53.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:53.691+0000 7f72b375c700 1 --2- 172.21.15.32:0/1577322762 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72ac102d50 0x7f72ac10f240 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:53.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:53.691+0000 7f72b375c700 1 --2- 172.21.15.32:0/1577322762 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72ac108400 0x7f72ac1087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:53.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:53.691+0000 7f72b375c700 1 --2- 172.21.15.32:0/1577322762 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f72ac102390 0x7f72ac102810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:53.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:53.691+0000 7f72b375c700 1 -- 172.21.15.32:0/1577322762 >> 172.21.15.32:0/1577322762 conn(0x7f72ac079310 msgr2=0x7f72ac079720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:46:53.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:53.691+0000 7f72b375c700 1 -- 172.21.15.32:0/1577322762 shutdown_connections 2024-05-03T18:46:53.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:53.691+0000 7f72b375c700 1 -- 172.21.15.32:0/1577322762 wait complete. 2024-05-03T18:46:53.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:53.692+0000 7f72b375c700 1 Processor -- start 2024-05-03T18:46:53.693 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:53.692+0000 7f72b375c700 1 -- start start 2024-05-03T18:46:53.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:53.692+0000 7f72b375c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72ac102390 0x7f72ac19f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:53.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:53.692+0000 7f72b375c700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f72ac102d50 0x7f72ac19fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:53.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:53.692+0000 7f72b375c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72ac108400 0x7f72ac199a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:53.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:53.692+0000 7f72b375c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f72ac112620 con 0x7f72ac108400 2024-05-03T18:46:53.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:53.692+0000 7f72b375c700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f72ac1124a0 con 0x7f72ac102390 2024-05-03T18:46:53.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:53.692+0000 7f72b375c700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f72ac1127a0 con 0x7f72ac102d50 2024-05-03T18:46:53.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:53.692+0000 7f72b1cf9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72ac108400 0x7f72ac199a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:53.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:53.692+0000 7f72b1cf9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72ac108400 0x7f72ac199a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58142/0 (socket says 172.21.15.32:58142) 2024-05-03T18:46:53.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:53.692+0000 7f72b1cf9700 1 -- 172.21.15.32:0/2690663407 learned_addr learned my addr 172.21.15.32:0/2690663407 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:46:53.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:53.692+0000 7f72b14f8700 1 --2- 172.21.15.32:0/2690663407 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72ac102390 0x7f72ac19f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:53.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:53.692+0000 7f72b0cf7700 1 --2- 172.21.15.32:0/2690663407 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f72ac102d50 0x7f72ac19fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:53.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:53.692+0000 7f72b1cf9700 1 -- 172.21.15.32:0/2690663407 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f72ac102d50 msgr2=0x7f72ac19fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:53.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:53.692+0000 7f72b1cf9700 1 --2- 172.21.15.32:0/2690663407 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f72ac102d50 0x7f72ac19fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:53.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:53.692+0000 7f72b1cf9700 1 -- 172.21.15.32:0/2690663407 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72ac102390 msgr2=0x7f72ac19f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:53.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:53.692+0000 7f72b1cf9700 1 --2- 172.21.15.32:0/2690663407 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72ac102390 0x7f72ac19f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:53.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:53.692+0000 7f72b1cf9700 1 -- 172.21.15.32:0/2690663407 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72ac19a1d0 con 0x7f72ac108400 2024-05-03T18:46:53.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:53.692+0000 7f72b1cf9700 1 --2- 172.21.15.32:0/2690663407 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72ac108400 0x7f72ac199a70 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f72a40062a0 tx=0x7f72a4004e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:46:53.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:53.693+0000 7f72a27fc700 1 -- 172.21.15.32:0/2690663407 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72a4021070 con 0x7f72ac108400 2024-05-03T18:46:53.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:53.693+0000 7f72a27fc700 1 -- 172.21.15.32:0/2690663407 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f72a4007c80 con 0x7f72ac108400 2024-05-03T18:46:53.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:53.693+0000 7f72b375c700 1 -- 172.21.15.32:0/2690663407 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72ac19a460 con 0x7f72ac108400 2024-05-03T18:46:53.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:53.693+0000 7f72a27fc700 1 -- 172.21.15.32:0/2690663407 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72a401d450 con 0x7f72ac108400 2024-05-03T18:46:53.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:53.693+0000 7f72b375c700 1 -- 172.21.15.32:0/2690663407 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72ac1a6780 con 0x7f72ac108400 2024-05-03T18:46:53.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:53.694+0000 7f72b375c700 1 -- 172.21.15.32:0/2690663407 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72ac06a190 con 0x7f72ac108400 2024-05-03T18:46:53.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:53.695+0000 7f72a27fc700 1 -- 172.21.15.32:0/2690663407 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f72a4004000 con 0x7f72ac108400 2024-05-03T18:46:53.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:53.696+0000 7f72a27fc700 1 --2- 172.21.15.32:0/2690663407 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f72980b2910 0x7f72980b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:53.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:53.696+0000 7f72b14f8700 1 --2- 172.21.15.32:0/2690663407 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f72980b2910 0x7f72980b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:53.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:53.696+0000 7f72a27fc700 1 -- 172.21.15.32:0/2690663407 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8356+0+0 (secure 0 0 0) 0x7f72a400f020 con 0x7f72ac108400 2024-05-03T18:46:53.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:53.696+0000 7f72b14f8700 1 --2- 172.21.15.32:0/2690663407 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f72980b2910 0x7f72980b4dd0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f72ac19abb0 tx=0x7f729c009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:46:53.700 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:53.699+0000 7f72a27fc700 1 -- 172.21.15.32:0/2690663407 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f72a402a020 con 0x7f72ac108400 2024-05-03T18:46:53.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:53.916+0000 7f72b375c700 1 -- 172.21.15.32:0/2690663407 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f72ac1a6950 con 0x7f72ac108400 2024-05-03T18:46:53.918 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:53.917+0000 7f72a27fc700 1 -- 172.21.15.32:0/2690663407 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+22814 (secure 0 0 0) 0x7f72a40abaf0 con 0x7f72ac108400 2024-05-03T18:46:53.918 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-03T18:46:53.919 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":82,"fsid":"f500455e-097b-11ef-bc97-c7b262605968","created":"2024-05-03T18:36:43.750772+0000","modified":"2024-05-03T18:46:47.787519+0000","last_up_change":"2024-05-03T18:43:50.933818+0000","last_in_change":"2024-05-03T18:43:31.804386+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-03T18:40:07.981853+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-03T18:44:44.607622+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-03T18:44:48.077391+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0430c3c0-2c50-471c-9c74-2869a0627c74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":262324956},{"type":"v1","addr":"172.21.15.32:6803","nonce":262324956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":262324956},{"type":"v1","addr":"172.21.15.32:6805","nonce":262324956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":262324956},{"type":"v1","addr":"172.21.15.32:6809","nonce":262324956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":262324956},{"type":"v1","addr":"172.21.15.32:6807","nonce":262324956}]},"public_addr":"172.21.15.32:6803/262324956","cluster_addr":"172.21.15.32:6805/262324956","heartbeat_back_addr":"172.21.15.32:6809/262324956","heartbeat_front_addr":"172.21.15.32:6807/262324956","state":["exists","up"]},{"osd":1,"uuid":"ba8f0b67-1d7d-4d4c-8276-eff664d21f02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1209907302},{"type":"v1","addr":"172.21.15.45:6801","nonce":1209907302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1209907302},{"type":"v1","addr":"172.21.15.45:6803","nonce":1209907302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1209907302},{"type":"v1","addr":"172.21.15.45:6807","nonce":1209907302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1209907302},{"type":"v1","addr":"172.21.15.45:6805","nonce":1209907302}]},"public_addr":"172.21.15.45:6801/1209907302","cluster_addr":"172.21.15.45:6803/1209907302","heartbeat_back_addr":"172.21.15.45:6807/1209907302","heartbeat_front_addr":"172.21.15.45:6805/1209907302","state":["exists","up"]},{"osd":2,"uuid":"5624eabc-be47-4bb8-9d8e-b84734769c83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":2308128926},{"type":"v1","addr":"172.21.15.113:6801","nonce":2308128926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":2308128926},{"type":"v1","addr":"172.21.15.113:6803","nonce":2308128926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":2308128926},{"type":"v1","addr":"172.21.15.113:6807","nonce":2308128926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":2308128926},{"type":"v1","addr":"172.21.15.113:6805","nonce":2308128926}]},"public_addr":"172.21.15.113:6801/2308128926","cluster_addr":"172.21.15.113:6803/2308128926","heartbeat_back_addr":"172.21.15.113:6807/2308128926","heartbeat_front_addr":"172.21.15.113:6805/2308128926","state":["exists","up"]},{"osd":3,"uuid":"62c9c977-afe1-4112-beba-b25af7caa2f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":2606514063},{"type":"v1","addr":"172.21.15.32:6811","nonce":2606514063}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":2606514063},{"type":"v1","addr":"172.21.15.32:6813","nonce":2606514063}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":2606514063},{"type":"v1","addr":"172.21.15.32:6817","nonce":2606514063}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":2606514063},{"type":"v1","addr":"172.21.15.32:6815","nonce":2606514063}]},"public_addr":"172.21.15.32:6811/2606514063","cluster_addr":"172.21.15.32:6813/2606514063","heartbeat_back_addr":"172.21.15.32:6817/2606514063","heartbeat_front_addr":"172.21.15.32:6815/2606514063","state":["exists","up"]},{"osd":4,"uuid":"99ad449f-b0bc-4aa1-b0cf-4a7a458126e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":71932118},{"type":"v1","addr":"172.21.15.45:6809","nonce":71932118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":71932118},{"type":"v1","addr":"172.21.15.45:6811","nonce":71932118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":71932118},{"type":"v1","addr":"172.21.15.45:6815","nonce":71932118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":71932118},{"type":"v1","addr":"172.21.15.45:6813","nonce":71932118}]},"public_addr":"172.21.15.45:6809/71932118","cluster_addr":"172.21.15.45:6811/71932118","heartbeat_back_addr":"172.21.15.45:6815/71932118","heartbeat_front_addr":"172.21.15.45:6813/71932118","state":["exists","up"]},{"osd":5,"uuid":"a6688ade-47d7-4a52-807f-3c880a8ffcbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":4085018597},{"type":"v1","addr":"172.21.15.113:6809","nonce":4085018597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":4085018597},{"type":"v1","addr":"172.21.15.113:6811","nonce":4085018597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":4085018597},{"type":"v1","addr":"172.21.15.113:6815","nonce":4085018597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":4085018597},{"type":"v1","addr":"172.21.15.113:6813","nonce":4085018597}]},"public_addr":"172.21.15.113:6809/4085018597","cluster_addr":"172.21.15.113:6811/4085018597","heartbeat_back_addr":"172.21.15.113:6815/4085018597","heartbeat_front_addr":"172.21.15.113:6813/4085018597","state":["exists","up"]},{"osd":6,"uuid":"1d9ebda9-55c7-473a-9dc0-fc60845bbfe9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":791407796},{"type":"v1","addr":"172.21.15.32:6819","nonce":791407796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":791407796},{"type":"v1","addr":"172.21.15.32:6821","nonce":791407796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":791407796},{"type":"v1","addr":"172.21.15.32:6825","nonce":791407796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":791407796},{"type":"v1","addr":"172.21.15.32:6823","nonce":791407796}]},"public_addr":"172.21.15.32:6819/791407796","cluster_addr":"172.21.15.32:6821/791407796","heartbeat_back_addr":"172.21.15.32:6825/791407796","heartbeat_front_addr":"172.21.15.32:6823/791407796","state":["exists","up"]},{"osd":7,"uuid":"988cce65-23ed-4814-ac09-177eb2f89fe1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1032572438},{"type":"v1","addr":"172.21.15.45:6817","nonce":1032572438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1032572438},{"type":"v1","addr":"172.21.15.45:6819","nonce":1032572438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1032572438},{"type":"v1","addr":"172.21.15.45:6823","nonce":1032572438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1032572438},{"type":"v1","addr":"172.21.15.45:6821","nonce":1032572438}]},"public_addr":"172.21.15.45:6817/1032572438","cluster_addr":"172.21.15.45:6819/1032572438","heartbeat_back_addr":"172.21.15.45:6823/1032572438","heartbeat_front_addr":"172.21.15.45:6821/1032572438","state":["exists","up"]},{"osd":8,"uuid":"816e8ce8-5afa-4b52-89be-f7793ed12a0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3569181558},{"type":"v1","addr":"172.21.15.113:6817","nonce":3569181558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3569181558},{"type":"v1","addr":"172.21.15.113:6819","nonce":3569181558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3569181558},{"type":"v1","addr":"172.21.15.113:6823","nonce":3569181558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3569181558},{"type":"v1","addr":"172.21.15.113:6821","nonce":3569181558}]},"public_addr":"172.21.15.113:6817/3569181558","cluster_addr":"172.21.15.113:6819/3569181558","heartbeat_back_addr":"172.21.15.113:6823/3569181558","heartbeat_front_addr":"172.21.15.113:6821/3569181558","state":["exists","up"]},{"osd":9,"uuid":"1d35c435-36b8-40e8-a2d4-0efe904f1107","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":4290196345},{"type":"v1","addr":"172.21.15.32:6827","nonce":4290196345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":4290196345},{"type":"v1","addr":"172.21.15.32:6829","nonce":4290196345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":4290196345},{"type":"v1","addr":"172.21.15.32:6833","nonce":4290196345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":4290196345},{"type":"v1","addr":"172.21.15.32:6831","nonce":4290196345}]},"public_addr":"172.21.15.32:6827/4290196345","cluster_addr":"172.21.15.32:6829/4290196345","heartbeat_back_addr":"172.21.15.32:6833/4290196345","heartbeat_front_addr":"172.21.15.32:6831/4290196345","state":["exists","up"]},{"osd":10,"uuid":"9af7879e-aa8d-4ef8-b289-4b145b614820","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1817064361},{"type":"v1","addr":"172.21.15.45:6825","nonce":1817064361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1817064361},{"type":"v1","addr":"172.21.15.45:6827","nonce":1817064361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1817064361},{"type":"v1","addr":"172.21.15.45:6831","nonce":1817064361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1817064361},{"type":"v1","addr":"172.21.15.45:6829","nonce":1817064361}]},"public_addr":"172.21.15.45:6825/1817064361","cluster_addr":"172.21.15.45:6827/1817064361","heartbeat_back_addr":"172.21.15.45:6831/1817064361","heartbeat_front_addr":"172.21.15.45:6829/1817064361","state":["exists","up"]},{"osd":11,"uuid":"22981709-1f9d-46f6-892a-5678394a2e1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":703464306},{"type":"v1","addr":"172.21.15.113:6825","nonce":703464306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":703464306},{"type":"v1","addr":"172.21.15.113:6827","nonce":703464306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":703464306},{"type":"v1","addr":"172.21.15.113:6831","nonce":703464306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":703464306},{"type":"v1","addr":"172.21.15.113:6829","nonce":703464306}]},"public_addr":"172.21.15.113:6825/703464306","cluster_addr":"172.21.15.113:6827/703464306","heartbeat_back_addr":"172.21.15.113:6831/703464306","heartbeat_front_addr":"172.21.15.113:6829/703464306","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:39:18.455575+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:39:42.234466+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:40:05.843072+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:40:30.822533+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:40:54.952487+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:41:18.750184+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:41:43.403326+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:42:08.285730+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:42:32.861740+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:42:59.307277+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:43:24.509691+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:43:49.850077+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":2,"to":1}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.c","mappings":[{"from":5,"to":8}]},{"pgid":"3.e","mappings":[{"from":4,"to":7}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.2f","mappings":[{"from":1,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"3.3b","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:0/743758274":"2024-05-04T18:37:45.879602+0000","172.21.15.32:6801/2019702098":"2024-05-04T18:37:45.879602+0000","172.21.15.32:6800/2019702098":"2024-05-04T18:37:45.879602+0000","172.21.15.32:0/3192385439":"2024-05-04T18:37:18.492319+0000","172.21.15.32:0/1450308494":"2024-05-04T18:37:18.492319+0000","172.21.15.32:0/3957699240":"2024-05-04T18:37:45.879602+0000","172.21.15.32:0/3310503393":"2024-05-04T18:37:18.492319+0000","172.21.15.32:6801/3945405684":"2024-05-04T18:37:18.492319+0000","172.21.15.32:6800/3945405684":"2024-05-04T18:37:18.492319+0000","172.21.15.32:0/2471164468":"2024-05-04T18:37:45.879602+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-03T18:46:53.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:53.919+0000 7f72b375c700 1 -- 172.21.15.32:0/2690663407 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f72980b2910 msgr2=0x7f72980b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:53.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:53.919+0000 7f72b375c700 1 --2- 172.21.15.32:0/2690663407 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f72980b2910 0x7f72980b4dd0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f72ac19abb0 tx=0x7f729c009190 comp rx=0 tx=0).stop 2024-05-03T18:46:53.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:53.919+0000 7f72b375c700 1 -- 172.21.15.32:0/2690663407 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72ac108400 msgr2=0x7f72ac199a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:53.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:53.919+0000 7f72b375c700 1 --2- 172.21.15.32:0/2690663407 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72ac108400 0x7f72ac199a70 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f72a40062a0 tx=0x7f72a4004e20 comp rx=0 tx=0).stop 2024-05-03T18:46:53.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:53.919+0000 7f72b375c700 1 -- 172.21.15.32:0/2690663407 shutdown_connections 2024-05-03T18:46:53.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:53.919+0000 7f72b375c700 1 --2- 172.21.15.32:0/2690663407 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f72980b2910 0x7f72980b4dd0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:53.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:53.919+0000 7f72b375c700 1 --2- 172.21.15.32:0/2690663407 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72ac108400 0x7f72ac199a70 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:53.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:53.919+0000 7f72b375c700 1 --2- 172.21.15.32:0/2690663407 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72ac102390 0x7f72ac19f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:53.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:53.919+0000 7f72b375c700 1 --2- 172.21.15.32:0/2690663407 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f72ac102d50 0x7f72ac19fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:53.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:53.919+0000 7f72b375c700 1 -- 172.21.15.32:0/2690663407 >> 172.21.15.32:0/2690663407 conn(0x7f72ac079310 msgr2=0x7f72ac1016c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:46:53.921 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:53.920+0000 7f72b375c700 1 -- 172.21.15.32:0/2690663407 shutdown_connections 2024-05-03T18:46:53.922 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:53.920+0000 7f72b375c700 1 -- 172.21.15.32:0/2690663407 wait complete. 2024-05-03T18:46:54.249 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:46:54 smithi032 ceph-mon[29763]: from='client.? 172.21.15.32:0/2690663407' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T18:46:54.497 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:46:54 smithi045 ceph-mon[33299]: from='client.? 172.21.15.32:0/2690663407' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T18:46:54.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:46:54 smithi113 ceph-mon[33540]: from='client.? 172.21.15.32:0/2690663407' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T18:46:54.612 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid f500455e-097b-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-03T18:46:54.862 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:46:55.295 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:46:55 smithi032 ceph-mon[29763]: pgmap v346: 97 pgs: 97 active+clean; 631 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 127 B/s wr, 6 op/s; 75 B/s, 1 keys/s, 0 objects/s recovering 2024-05-03T18:46:55.452 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:46:55 smithi113 ceph-mon[33540]: pgmap v346: 97 pgs: 97 active+clean; 631 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 127 B/s wr, 6 op/s; 75 B/s, 1 keys/s, 0 objects/s recovering 2024-05-03T18:46:55.497 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:46:55 smithi045 ceph-mon[33299]: pgmap v346: 97 pgs: 97 active+clean; 631 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 127 B/s wr, 6 op/s; 75 B/s, 1 keys/s, 0 objects/s recovering 2024-05-03T18:46:56.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:56.482+0000 7fc01e9ce700 1 -- 172.21.15.32:0/2609050707 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc018075ff0 msgr2=0x7fc018114130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:56.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:56.482+0000 7fc01e9ce700 1 --2- 172.21.15.32:0/2609050707 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc018075ff0 0x7fc018114130 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fc000009990 tx=0x7fc000009ca0 comp rx=0 tx=0).stop 2024-05-03T18:46:56.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:56.482+0000 7fc01e9ce700 1 -- 172.21.15.32:0/2609050707 shutdown_connections 2024-05-03T18:46:56.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:56.482+0000 7fc01e9ce700 1 --2- 172.21.15.32:0/2609050707 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc018114670 0x7fc018116a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:56.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:56.482+0000 7fc01e9ce700 1 --2- 172.21.15.32:0/2609050707 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc0180756d0 0x7fc018075ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:56.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:56.482+0000 7fc01e9ce700 1 --2- 172.21.15.32:0/2609050707 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc018075ff0 0x7fc018114130 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:56.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:56.482+0000 7fc01e9ce700 1 -- 172.21.15.32:0/2609050707 >> 172.21.15.32:0/2609050707 conn(0x7fc01807a600 msgr2=0x7fc01807aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:46:56.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:56.482+0000 7fc01e9ce700 1 -- 172.21.15.32:0/2609050707 shutdown_connections 2024-05-03T18:46:56.484 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:56.482+0000 7fc01e9ce700 1 -- 172.21.15.32:0/2609050707 wait complete. 2024-05-03T18:46:56.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:56.483+0000 7fc01e9ce700 1 Processor -- start 2024-05-03T18:46:56.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:56.483+0000 7fc01e9ce700 1 -- start start 2024-05-03T18:46:56.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:56.483+0000 7fc01e9ce700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc0180756d0 0x7fc0181a0190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:56.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:56.483+0000 7fc01e9ce700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc018075ff0 0x7fc0181a06d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:56.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:56.483+0000 7fc01e9ce700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc018114670 0x7fc0181a4a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:56.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:56.483+0000 7fc01e9ce700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fc018119c80 con 0x7fc018114670 2024-05-03T18:46:56.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:56.483+0000 7fc01e9ce700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc018119b00 con 0x7fc018075ff0 2024-05-03T18:46:56.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:56.483+0000 7fc01e9ce700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fc018119e00 con 0x7fc0180756d0 2024-05-03T18:46:56.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:56.483+0000 7fc01cf6b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc018114670 0x7fc0181a4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:56.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:56.483+0000 7fc01cf6b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc018114670 0x7fc0181a4a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58164/0 (socket says 172.21.15.32:58164) 2024-05-03T18:46:56.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:56.483+0000 7fc01cf6b700 1 -- 172.21.15.32:0/1858939979 learned_addr learned my addr 172.21.15.32:0/1858939979 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:46:56.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:56.483+0000 7fc0177fe700 1 --2- 172.21.15.32:0/1858939979 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc018075ff0 0x7fc0181a06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:56.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:56.483+0000 7fc017fff700 1 --2- 172.21.15.32:0/1858939979 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc0180756d0 0x7fc0181a0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:56.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:56.483+0000 7fc01cf6b700 1 -- 172.21.15.32:0/1858939979 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc0180756d0 msgr2=0x7fc0181a0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:56.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:56.483+0000 7fc01cf6b700 1 --2- 172.21.15.32:0/1858939979 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc0180756d0 0x7fc0181a0190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:56.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:56.484+0000 7fc01cf6b700 1 -- 172.21.15.32:0/1858939979 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc018075ff0 msgr2=0x7fc0181a06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:56.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:56.484+0000 7fc01cf6b700 1 --2- 172.21.15.32:0/1858939979 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc018075ff0 0x7fc0181a06d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:56.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:56.484+0000 7fc01cf6b700 1 -- 172.21.15.32:0/1858939979 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0181a51a0 con 0x7fc018114670 2024-05-03T18:46:56.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:56.484+0000 7fc017fff700 1 --2- 172.21.15.32:0/1858939979 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc0180756d0 0x7fc0181a0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T18:46:56.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:56.484+0000 7fc01cf6b700 1 --2- 172.21.15.32:0/1858939979 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc018114670 0x7fc0181a4a60 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fc01000ef90 tx=0x7fc01000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:46:56.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:56.484+0000 7fc0157fa700 1 -- 172.21.15.32:0/1858939979 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0100092e0 con 0x7fc018114670 2024-05-03T18:46:56.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:56.484+0000 7fc01e9ce700 1 -- 172.21.15.32:0/1858939979 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0181a5430 con 0x7fc018114670 2024-05-03T18:46:56.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:56.484+0000 7fc0157fa700 1 -- 172.21.15.32:0/1858939979 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc010004bb0 con 0x7fc018114670 2024-05-03T18:46:56.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:56.484+0000 7fc0157fa700 1 -- 172.21.15.32:0/1858939979 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc010004280 con 0x7fc018114670 2024-05-03T18:46:56.487 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:56.484+0000 7fc01e9ce700 1 -- 172.21.15.32:0/1858939979 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0181acd10 con 0x7fc018114670 2024-05-03T18:46:56.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:56.485+0000 7fc01e9ce700 1 -- 172.21.15.32:0/1858939979 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc01806a190 con 0x7fc018114670 2024-05-03T18:46:56.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:56.486+0000 7fc0157fa700 1 -- 172.21.15.32:0/1858939979 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fc01001b030 con 0x7fc018114670 2024-05-03T18:46:56.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:56.487+0000 7fc0157fa700 1 --2- 172.21.15.32:0/1858939979 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fc0040b2910 0x7fc0040b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:56.488 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:56.487+0000 7fc017fff700 1 --2- 172.21.15.32:0/1858939979 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fc0040b2910 0x7fc0040b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:56.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:56.487+0000 7fc0157fa700 1 -- 172.21.15.32:0/1858939979 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8356+0+0 (secure 0 0 0) 0x7fc0100e2910 con 0x7fc018114670 2024-05-03T18:46:56.489 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:56.488+0000 7fc017fff700 1 --2- 172.21.15.32:0/1858939979 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fc0040b2910 0x7fc0040b4dd0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fc00800bc10 tx=0x7fc00800b5f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:46:56.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:56.491+0000 7fc0157fa700 1 -- 172.21.15.32:0/1858939979 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fc010022020 con 0x7fc018114670 2024-05-03T18:46:56.754 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:56.753+0000 7fc01e9ce700 1 -- 172.21.15.32:0/1858939979 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc0181acfa0 con 0x7fc018114670 2024-05-03T18:46:56.755 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:56.754+0000 7fc0157fa700 1 -- 172.21.15.32:0/1858939979 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11862 (secure 0 0 0) 0x7fc010010040 con 0x7fc018114670 2024-05-03T18:46:56.755 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-03T18:46:56.755 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/535599929","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":535599929},{"type":"v1","addr":"172.21.15.32:6835","nonce":535599929}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/3101679868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":3101679868},{"type":"v1","addr":"172.21.15.32:6837","nonce":3101679868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14610,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/3838653921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":3838653921},{"type":"v1","addr":"172.21.15.32:6839","nonce":3838653921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14622,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6841/3019648436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6840","nonce":3019648436},{"type":"v1","addr":"172.21.15.32:6841","nonce":3019648436}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24380,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/1629869435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":1629869435},{"type":"v1","addr":"172.21.15.45:6833","nonce":1629869435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24398,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2997291166","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2997291166},{"type":"v1","addr":"172.21.15.45:6835","nonce":2997291166}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-03T18:44:51.155050+0000","modified":"2024-05-03T18:45:08.404530+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24478,"mds_1":24469,"mds_2":24463},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24425":{"gid":24425,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.45:6837/428055392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":428055392},{"type":"v1","addr":"172.21.15.45:6837","nonce":428055392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.113:6833/3323558791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":3323558791},{"type":"v1","addr":"172.21.15.113:6833","nonce":3323558791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"b","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.45:6839/970804040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":970804040},{"type":"v1","addr":"172.21.15.45:6839","nonce":970804040}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.113:6835/2588738276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2588738276},{"type":"v1","addr":"172.21.15.113:6835","nonce":2588738276}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.113:6837/3677284460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":3677284460},{"type":"v1","addr":"172.21.15.113:6837","nonce":3677284460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.113:6839/942050693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6838","nonce":942050693},{"type":"v1","addr":"172.21.15.113:6839","nonce":942050693}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24478,"qdb_cluster":[24469,24478,24463]},"id":1}]} 2024-05-03T18:46:56.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:56.756+0000 7fc01e9ce700 1 -- 172.21.15.32:0/1858939979 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fc0040b2910 msgr2=0x7fc0040b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:56.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:56.756+0000 7fc01e9ce700 1 --2- 172.21.15.32:0/1858939979 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fc0040b2910 0x7fc0040b4dd0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fc00800bc10 tx=0x7fc00800b5f0 comp rx=0 tx=0).stop 2024-05-03T18:46:56.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:56.756+0000 7fc01e9ce700 1 -- 172.21.15.32:0/1858939979 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc018114670 msgr2=0x7fc0181a4a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:56.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:56.756+0000 7fc01e9ce700 1 --2- 172.21.15.32:0/1858939979 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc018114670 0x7fc0181a4a60 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fc01000ef90 tx=0x7fc01000c3a0 comp rx=0 tx=0).stop 2024-05-03T18:46:56.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:56.756+0000 7fc01e9ce700 1 -- 172.21.15.32:0/1858939979 shutdown_connections 2024-05-03T18:46:56.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:56.756+0000 7fc01e9ce700 1 --2- 172.21.15.32:0/1858939979 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fc0040b2910 0x7fc0040b4dd0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:56.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:56.756+0000 7fc01e9ce700 1 --2- 172.21.15.32:0/1858939979 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc018114670 0x7fc0181a4a60 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:56.757 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:56.756+0000 7fc01e9ce700 1 --2- 172.21.15.32:0/1858939979 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc0180756d0 0x7fc0181a0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:56.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:56.756+0000 7fc01e9ce700 1 --2- 172.21.15.32:0/1858939979 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc018075ff0 0x7fc0181a06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:56.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:56.756+0000 7fc01e9ce700 1 -- 172.21.15.32:0/1858939979 >> 172.21.15.32:0/1858939979 conn(0x7fc01807a600 msgr2=0x7fc0181134a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:46:56.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:56.756+0000 7fc01e9ce700 1 -- 172.21.15.32:0/1858939979 shutdown_connections 2024-05-03T18:46:56.758 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:56.756+0000 7fc01e9ce700 1 -- 172.21.15.32:0/1858939979 wait complete. 2024-05-03T18:46:56.758 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 25 2024-05-03T18:46:57.315 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:46:57 smithi032 ceph-mon[29763]: pgmap v347: 97 pgs: 97 active+clean; 631 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 111 B/s wr, 6 op/s; 65 B/s, 1 keys/s, 0 objects/s recovering 2024-05-03T18:46:57.315 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:46:57 smithi032 ceph-mon[29763]: from='client.? 172.21.15.32:0/1858939979' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:46:57.345 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 25, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-05-03T18:44:51.155050+0000', 'modified': '2024-05-03T18:45:08.404530+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24478, 'mds_1': 24469, 'mds_2': 24463}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24425': {'gid': 24425, 'name': 'h', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.45:6837/428055392', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6836', 'nonce': 428055392}, {'type': 'v1', 'addr': '172.21.15.45:6837', 'nonce': 428055392}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24430': {'gid': 24430, 'name': 'l', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.113:6833/3323558791', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.113:6832', 'nonce': 3323558791}, {'type': 'v1', 'addr': '172.21.15.113:6833', 'nonce': 3323558791}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24446': {'gid': 24446, 'name': 'b', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.45:6839/970804040', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6838', 'nonce': 970804040}, {'type': 'v1', 'addr': '172.21.15.45:6839', 'nonce': 970804040}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24463': {'gid': 24463, 'name': 'f', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.113:6835/2588738276', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.113:6834', 'nonce': 2588738276}, {'type': 'v1', 'addr': '172.21.15.113:6835', 'nonce': 2588738276}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24469': {'gid': 24469, 'name': 'c', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.113:6837/3677284460', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.113:6836', 'nonce': 3677284460}, {'type': 'v1', 'addr': '172.21.15.113:6837', 'nonce': 3677284460}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24478': {'gid': 24478, 'name': 'i', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.113:6839/942050693', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.113:6838', 'nonce': 942050693}, {'type': 'v1', 'addr': '172.21.15.113:6839', 'nonce': 942050693}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24478, 'qdb_cluster': [24469, 24478, 24463]} 2024-05-03T18:46:57.345 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-05-03T18:46:57.345 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid f500455e-097b-11ef-bc97-c7b262605968 -- ceph tell mds.f status 2024-05-03T18:46:57.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:46:57 smithi045 ceph-mon[33299]: pgmap v347: 97 pgs: 97 active+clean; 631 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 111 B/s wr, 6 op/s; 65 B/s, 1 keys/s, 0 objects/s recovering 2024-05-03T18:46:57.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:46:57 smithi045 ceph-mon[33299]: from='client.? 172.21.15.32:0/1858939979' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:46:57.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:46:57 smithi113 ceph-mon[33540]: pgmap v347: 97 pgs: 97 active+clean; 631 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 111 B/s wr, 6 op/s; 65 B/s, 1 keys/s, 0 objects/s recovering 2024-05-03T18:46:57.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:46:57 smithi113 ceph-mon[33540]: from='client.? 172.21.15.32:0/1858939979' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:46:57.594 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:46:59.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.234+0000 7f7f1f3d9700 1 -- 172.21.15.32:0/2777234434 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f1806c630 msgr2=0x7f7f18110360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:59.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.234+0000 7f7f1f3d9700 1 --2- 172.21.15.32:0/2777234434 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f1806c630 0x7f7f18110360 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f7f1000b0d0 tx=0x7f7f1000b3e0 comp rx=0 tx=0).stop 2024-05-03T18:46:59.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.234+0000 7f7f1f3d9700 1 -- 172.21.15.32:0/2777234434 shutdown_connections 2024-05-03T18:46:59.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.234+0000 7f7f1f3d9700 1 --2- 172.21.15.32:0/2777234434 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f1806c630 0x7f7f18110360 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:59.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.234+0000 7f7f1f3d9700 1 --2- 172.21.15.32:0/2777234434 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7f18109590 0x7f7f18109970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:59.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.234+0000 7f7f1f3d9700 1 --2- 172.21.15.32:0/2777234434 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7f1806bc70 0x7f7f1806c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:59.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.234+0000 7f7f1f3d9700 1 -- 172.21.15.32:0/2777234434 >> 172.21.15.32:0/2777234434 conn(0x7f7f18079440 msgr2=0x7f7f18079850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:46:59.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.234+0000 7f7f1f3d9700 1 -- 172.21.15.32:0/2777234434 shutdown_connections 2024-05-03T18:46:59.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.234+0000 7f7f1f3d9700 1 -- 172.21.15.32:0/2777234434 wait complete. 2024-05-03T18:46:59.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.235+0000 7f7f1f3d9700 1 Processor -- start 2024-05-03T18:46:59.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.235+0000 7f7f1f3d9700 1 -- start start 2024-05-03T18:46:59.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.235+0000 7f7f1f3d9700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f1806bc70 0x7f7f18102410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:59.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.235+0000 7f7f1f3d9700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7f1806c630 0x7f7f18102950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:59.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.235+0000 7f7f1f3d9700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7f18109590 0x7f7f18102e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:59.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.235+0000 7f7f1f3d9700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f7f18113690 con 0x7f7f1806bc70 2024-05-03T18:46:59.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.235+0000 7f7f1f3d9700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7f18113510 con 0x7f7f1806c630 2024-05-03T18:46:59.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.235+0000 7f7f1f3d9700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f7f18113810 con 0x7f7f18109590 2024-05-03T18:46:59.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.235+0000 7f7f1d175700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f1806bc70 0x7f7f18102410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:59.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.235+0000 7f7f1d175700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f1806bc70 0x7f7f18102410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58196/0 (socket says 172.21.15.32:58196) 2024-05-03T18:46:59.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.235+0000 7f7f1d175700 1 -- 172.21.15.32:0/2200040071 learned_addr learned my addr 172.21.15.32:0/2200040071 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:46:59.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.235+0000 7f7f1c974700 1 --2- 172.21.15.32:0/2200040071 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7f1806c630 0x7f7f18102950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:59.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.235+0000 7f7f1d976700 1 --2- 172.21.15.32:0/2200040071 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7f18109590 0x7f7f18102e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:59.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.235+0000 7f7f1d175700 1 -- 172.21.15.32:0/2200040071 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7f18109590 msgr2=0x7f7f18102e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:59.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.235+0000 7f7f1d175700 1 --2- 172.21.15.32:0/2200040071 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7f18109590 0x7f7f18102e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:59.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.235+0000 7f7f1d175700 1 -- 172.21.15.32:0/2200040071 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7f1806c630 msgr2=0x7f7f18102950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:59.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.235+0000 7f7f1d175700 1 --2- 172.21.15.32:0/2200040071 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7f1806c630 0x7f7f18102950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:59.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.235+0000 7f7f1d175700 1 -- 172.21.15.32:0/2200040071 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f181a7fc0 con 0x7f7f1806bc70 2024-05-03T18:46:59.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.235+0000 7f7f1d175700 1 --2- 172.21.15.32:0/2200040071 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f1806bc70 0x7f7f18102410 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f7f1400d980 tx=0x7f7f1400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:46:59.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.235+0000 7f7f0e7fc700 1 -- 172.21.15.32:0/2200040071 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f14016070 con 0x7f7f1806bc70 2024-05-03T18:46:59.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.236+0000 7f7f0e7fc700 1 -- 172.21.15.32:0/2200040071 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f14004830 con 0x7f7f1806bc70 2024-05-03T18:46:59.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.236+0000 7f7f0e7fc700 1 -- 172.21.15.32:0/2200040071 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f140054c0 con 0x7f7f1806bc70 2024-05-03T18:46:59.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.236+0000 7f7f1f3d9700 1 -- 172.21.15.32:0/2200040071 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f181a82b0 con 0x7f7f1806bc70 2024-05-03T18:46:59.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.236+0000 7f7f1f3d9700 1 -- 172.21.15.32:0/2200040071 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f181a8840 con 0x7f7f1806bc70 2024-05-03T18:46:59.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.238+0000 7f7f0e7fc700 1 -- 172.21.15.32:0/2200040071 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f7f14004220 con 0x7f7f1806bc70 2024-05-03T18:46:59.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.239+0000 7f7f0e7fc700 1 --2- 172.21.15.32:0/2200040071 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f7f040b2990 0x7f7f040b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:59.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.239+0000 7f7f1c974700 1 --2- 172.21.15.32:0/2200040071 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f7f040b2990 0x7f7f040b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:59.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.239+0000 7f7f0e7fc700 1 -- 172.21.15.32:0/2200040071 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8356+0+0 (secure 0 0 0) 0x7f7f140e21b0 con 0x7f7f1806bc70 2024-05-03T18:46:59.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.240+0000 7f7f29c6cb80 1 Processor -- start 2024-05-03T18:46:59.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.240+0000 7f7f29c6cb80 1 -- start start 2024-05-03T18:46:59.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.240+0000 7f7f1c974700 1 --2- 172.21.15.32:0/2200040071 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f7f040b2990 0x7f7f040b4e50 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f7f18079a40 tx=0x7f7f08009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:46:59.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.240+0000 7f7f29c6cb80 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d26f624740 0x55d26f624b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:59.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.240+0000 7f7f29c6cb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d26f625060 0x55d26f61ff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:59.246 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.240+0000 7f7f29c6cb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d26f620480 0x55d26f6208c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:59.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.240+0000 7f7f29c6cb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55d26f5728f0 con 0x55d26f620480 2024-05-03T18:46:59.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.240+0000 7f7f29c6cb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55d26f4d7d50 con 0x55d26f625060 2024-05-03T18:46:59.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.240+0000 7f7f29c6cb80 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55d26f56e8b0 con 0x55d26f624740 2024-05-03T18:46:59.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.240+0000 7f7f1d976700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d26f620480 0x55d26f6208c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:59.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.240+0000 7f7f1d976700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d26f620480 0x55d26f6208c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58210/0 (socket says 172.21.15.32:58210) 2024-05-03T18:46:59.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.240+0000 7f7f1c974700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d26f625060 0x55d26f61ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:59.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.240+0000 7f7f1d976700 1 -- 172.21.15.32:0/1728972630 learned_addr learned my addr 172.21.15.32:0/1728972630 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:46:59.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.240+0000 7f7f1d175700 1 --2- 172.21.15.32:0/1728972630 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d26f624740 0x55d26f624b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:59.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.240+0000 7f7f1d976700 1 -- 172.21.15.32:0/1728972630 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d26f624740 msgr2=0x55d26f624b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:46:59.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.240+0000 7f7f1d976700 1 --2- 172.21.15.32:0/1728972630 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d26f624740 0x55d26f624b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:59.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.240+0000 7f7f1d976700 1 -- 172.21.15.32:0/1728972630 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d26f625060 msgr2=0x55d26f61ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:46:59.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.240+0000 7f7f1d976700 1 --2- 172.21.15.32:0/1728972630 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d26f625060 0x55d26f61ff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:59.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.240+0000 7f7f1d976700 1 -- 172.21.15.32:0/1728972630 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d26f4bf880 con 0x55d26f620480 2024-05-03T18:46:59.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.241+0000 7f7f1d976700 1 --2- 172.21.15.32:0/1728972630 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d26f620480 0x55d26f6208c0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f7f10009dc0 tx=0x7f7f10009df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b3153f6ba3a5a1 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:46:59.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.241+0000 7f7f037fe700 1 -- 172.21.15.32:0/1728972630 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f10022070 con 0x55d26f620480 2024-05-03T18:46:59.247 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.241+0000 7f7f037fe700 1 -- 172.21.15.32:0/1728972630 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f100074e0 con 0x55d26f620480 2024-05-03T18:46:59.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.241+0000 7f7f037fe700 1 -- 172.21.15.32:0/1728972630 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f1001c6a0 con 0x55d26f620480 2024-05-03T18:46:59.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.241+0000 7f7f29c6cb80 1 -- 172.21.15.32:0/1728972630 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d26f620480 msgr2=0x55d26f6208c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:59.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.241+0000 7f7f29c6cb80 1 --2- 172.21.15.32:0/1728972630 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d26f620480 0x55d26f6208c0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f7f10009dc0 tx=0x7f7f10009df0 comp rx=0 tx=0).stop 2024-05-03T18:46:59.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.242+0000 7f7f29c6cb80 1 -- 172.21.15.32:0/1728972630 shutdown_connections 2024-05-03T18:46:59.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.242+0000 7f7f29c6cb80 1 --2- 172.21.15.32:0/1728972630 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d26f620480 0x55d26f6208c0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:59.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.242+0000 7f7f29c6cb80 1 --2- 172.21.15.32:0/1728972630 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d26f624740 0x55d26f624b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:59.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.242+0000 7f7f29c6cb80 1 --2- 172.21.15.32:0/1728972630 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d26f625060 0x55d26f61ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:59.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.242+0000 7f7f29c6cb80 1 -- 172.21.15.32:0/1728972630 >> 172.21.15.32:0/1728972630 conn(0x55d26f54cb20 msgr2=0x55d26f61a9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:46:59.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.242+0000 7f7f29c6cb80 1 -- 172.21.15.32:0/1728972630 shutdown_connections 2024-05-03T18:46:59.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.242+0000 7f7f29c6cb80 1 -- 172.21.15.32:0/1728972630 wait complete. 2024-05-03T18:46:59.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.242+0000 7f7f29c6cb80 1 -- start start 2024-05-03T18:46:59.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.242+0000 7f7f29c6cb80 1 Processor -- start 2024-05-03T18:46:59.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.242+0000 7f7f29c6cb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d26f620480 0x55d26f62bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:59.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.242+0000 7f7f29c6cb80 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d26f624740 0x55d26f6291b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:59.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.242+0000 7f7f29c6cb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d26f625060 0x55d26f6296f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:59.248 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.242+0000 7f7f29c6cb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55d26f62cb40 con 0x55d26f620480 2024-05-03T18:46:59.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.242+0000 7f7f29c6cb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55d26f52be80 con 0x55d26f625060 2024-05-03T18:46:59.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.242+0000 7f7f29c6cb80 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55d26f62ccc0 con 0x55d26f624740 2024-05-03T18:46:59.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.243+0000 7f7f29c6cb80 10 client.0 _ll_register_callbacks cb 0x55d26f506560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:46:59.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.243+0000 7f7f1d175700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d26f620480 0x55d26f62bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:59.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.243+0000 7f7f1d175700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d26f620480 0x55d26f62bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58214/0 (socket says 172.21.15.32:58214) 2024-05-03T18:46:59.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.243+0000 7f7f1d175700 1 -- 172.21.15.32:0/2866192282 learned_addr learned my addr 172.21.15.32:0/2866192282 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:46:59.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.243+0000 7f7f1d976700 1 --2- 172.21.15.32:0/2866192282 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d26f625060 0x55d26f6296f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:59.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.243+0000 7f7f1c974700 1 --2- 172.21.15.32:0/2866192282 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d26f624740 0x55d26f6291b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:59.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.243+0000 7f7f1d175700 1 -- 172.21.15.32:0/2866192282 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d26f624740 msgr2=0x55d26f6291b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:46:59.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.243+0000 7f7f1d175700 1 --2- 172.21.15.32:0/2866192282 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d26f624740 0x55d26f6291b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:59.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.243+0000 7f7f1d175700 1 -- 172.21.15.32:0/2866192282 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d26f625060 msgr2=0x55d26f6296f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:46:59.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.243+0000 7f7f1d175700 1 --2- 172.21.15.32:0/2866192282 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d26f625060 0x55d26f6296f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:59.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.243+0000 7f7f1d175700 1 -- 172.21.15.32:0/2866192282 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d26f529710 con 0x55d26f620480 2024-05-03T18:46:59.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.243+0000 7f7f1c974700 1 --2- 172.21.15.32:0/2866192282 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d26f624740 0x55d26f6291b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T18:46:59.249 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.243+0000 7f7f1d976700 1 --2- 172.21.15.32:0/2866192282 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d26f625060 0x55d26f6296f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T18:46:59.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.243+0000 7f7f1d175700 1 --2- 172.21.15.32:0/2866192282 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d26f620480 0x55d26f62bda0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f7f140dfab0 tx=0x7f7f14004f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=275457e1075a35ae server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:46:59.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.243+0000 7f7f00ff9700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-05-03T18:46:59.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.243+0000 7f7f00ff9700 1 -- 172.21.15.32:0/2866192282 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f140a9590 con 0x55d26f620480 2024-05-03T18:46:59.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.243+0000 7f7f00ff9700 1 -- 172.21.15.32:0/2866192282 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8356+0+0 (secure 0 0 0) 0x7f7f140a5b80 con 0x55d26f620480 2024-05-03T18:46:59.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.243+0000 7f7f29c6cb80 1 -- 172.21.15.32:0/2866192282 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f100074e0 con 0x55d26f620480 2024-05-03T18:46:59.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.244+0000 7f7f00ff9700 1 -- 172.21.15.32:0/2866192282 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7f140ca050 con 0x55d26f620480 2024-05-03T18:46:59.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.244+0000 7f7f29c6cb80 10 client.14976 fetch_fsmap learned FSMap version 25 2024-05-03T18:46:59.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.244+0000 7f7f29c6cb80 1 -- 172.21.15.32:0/2866192282 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d26f637380 con 0x55d26f620480 2024-05-03T18:46:59.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.244+0000 7f7f00ff9700 1 -- 172.21.15.32:0/2866192282 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f100074e0 con 0x55d26f620480 2024-05-03T18:46:59.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.244+0000 7f7f00ff9700 1 -- 172.21.15.32:0/2866192282 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f7f140a97f0 con 0x55d26f620480 2024-05-03T18:46:59.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.244+0000 7f7f29c6cb80 10 client.14976 fetch_fsmap finished waiting for FSMap version 25 2024-05-03T18:46:59.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.244+0000 7f7f29c6cb80 10 client.14976 resolve_mds: resolved name 'f' to daemon mds.f 2024-05-03T18:46:59.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.244+0000 7f7f29c6cb80 20 client.14976 populate_metadata read hostname 'smithi032' 2024-05-03T18:46:59.250 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.244+0000 7f7f29c6cb80 1 --2- 172.21.15.32:0/2866192282 >> [v2:172.21.15.113:6834/2588738276,v1:172.21.15.113:6835/2588738276] conn(0x55d26f6399b0 0x55d26f63ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:59.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.244+0000 7f7f29c6cb80 4 client.14976 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-03T18:46:59.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.244+0000 7f7f29c6cb80 1 -- 172.21.15.32:0/2866192282 --> [v2:172.21.15.113:6834/2588738276,v1:172.21.15.113:6835/2588738276] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d26f5802d0 con 0x55d26f6399b0 2024-05-03T18:46:59.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.244+0000 7f7f1c974700 1 --2- 172.21.15.32:0/2866192282 >> [v2:172.21.15.113:6834/2588738276,v1:172.21.15.113:6835/2588738276] conn(0x55d26f6399b0 0x55d26f63ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:59.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.245+0000 7f7f00ff9700 10 client.14976 ms_handle_connect on v2:172.21.15.113:6834/2588738276 2024-05-03T18:46:59.251 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.245+0000 7f7f1c974700 1 --2- 172.21.15.32:0/2866192282 >> [v2:172.21.15.113:6834/2588738276,v1:172.21.15.113:6835/2588738276] conn(0x55d26f6399b0 0x55d26f63ae20 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2d6a4d670656c6a8 server_cookie=23c171aa5315811d in_seq=0 out_seq=0 2024-05-03T18:46:59.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.245+0000 7f7f00ff9700 1 -- 172.21.15.32:0/2866192282 <== mds.2 v2:172.21.15.113:6834/2588738276 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d26f5802d0 con 0x55d26f6399b0 2024-05-03T18:46:59.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.245+0000 7f7f00ff9700 10 client.14976 handle_command_reply: tid=0 2024-05-03T18:46:59.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.245+0000 7f7f29c6cb80 1 client.14976 shutdown 2024-05-03T18:46:59.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.246+0000 7f7f29c6cb80 1 -- 172.21.15.32:0/2866192282 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d26f620480 msgr2=0x55d26f62bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:59.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.246+0000 7f7f29c6cb80 1 --2- 172.21.15.32:0/2866192282 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d26f620480 0x55d26f62bda0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f7f140dfab0 tx=0x7f7f14004f10 comp rx=0 tx=0).stop 2024-05-03T18:46:59.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.246+0000 7f7f29c6cb80 1 -- 172.21.15.32:0/2866192282 shutdown_connections 2024-05-03T18:46:59.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.246+0000 7f7f29c6cb80 1 --2- 172.21.15.32:0/2866192282 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d26f625060 0x55d26f6296f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:59.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.246+0000 7f7f29c6cb80 1 --2- 172.21.15.32:0/2866192282 >> [v2:172.21.15.113:6834/2588738276,v1:172.21.15.113:6835/2588738276] conn(0x55d26f6399b0 0x55d26f63ae20 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:59.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.246+0000 7f7f29c6cb80 1 --2- 172.21.15.32:0/2866192282 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d26f620480 0x55d26f62bda0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:59.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.246+0000 7f7f29c6cb80 1 --2- 172.21.15.32:0/2866192282 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d26f624740 0x55d26f6291b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:59.252 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.246+0000 7f7f29c6cb80 1 -- 172.21.15.32:0/2866192282 >> 172.21.15.32:0/2866192282 conn(0x55d26f54cb20 msgr2=0x55d26f619800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:46:59.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.246+0000 7f7f00ff9700 0 client.14976 ms_handle_reset on v2:172.21.15.113:6834/2588738276 2024-05-03T18:46:59.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.246+0000 7f7f29c6cb80 1 -- 172.21.15.32:0/2866192282 shutdown_connections 2024-05-03T18:46:59.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.246+0000 7f7f29c6cb80 1 -- 172.21.15.32:0/2866192282 wait complete. 2024-05-03T18:46:59.253 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.246+0000 7f7f29c6cb80 20 client.14976 trim_cache size 0 max 16384 2024-05-03T18:46:59.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.268+0000 7f7f29c6cb80 1 Processor -- start 2024-05-03T18:46:59.269 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.268+0000 7f7f29c6cb80 1 -- start start 2024-05-03T18:46:59.269 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:46:59 smithi032 ceph-mon[29763]: pgmap v348: 97 pgs: 97 active+clean; 631 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 100 B/s wr, 6 op/s; 59 B/s, 0 keys/s, 0 objects/s recovering 2024-05-03T18:46:59.270 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:46:59 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:46:59.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.268+0000 7f7f29c6cb80 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d26f620480 0x55d26f628730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:59.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.268+0000 7f7f29c6cb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d26f624740 0x55d26f629170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:59.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.268+0000 7f7f29c6cb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d26f625060 0x55d26f6296b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:59.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.268+0000 7f7f29c6cb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55d26f639160 con 0x55d26f625060 2024-05-03T18:46:59.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.268+0000 7f7f29c6cb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55d26f639830 con 0x55d26f624740 2024-05-03T18:46:59.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.268+0000 7f7f29c6cb80 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55d26f639f00 con 0x55d26f620480 2024-05-03T18:46:59.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.269+0000 7f7f1d175700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d26f620480 0x55d26f628730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:59.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.269+0000 7f7f1c974700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d26f624740 0x55d26f629170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:59.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.269+0000 7f7f1c974700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d26f624740 0x55d26f629170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.32:33326/0 (socket says 172.21.15.32:33326) 2024-05-03T18:46:59.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.269+0000 7f7f1c974700 1 -- 172.21.15.32:0/3010490739 learned_addr learned my addr 172.21.15.32:0/3010490739 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:46:59.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.269+0000 7f7f1d976700 1 --2- 172.21.15.32:0/3010490739 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d26f625060 0x55d26f6296b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:59.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.269+0000 7f7f1c974700 1 -- 172.21.15.32:0/3010490739 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d26f620480 msgr2=0x55d26f628730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:46:59.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.269+0000 7f7f1c974700 1 --2- 172.21.15.32:0/3010490739 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d26f620480 0x55d26f628730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:59.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.269+0000 7f7f1c974700 1 -- 172.21.15.32:0/3010490739 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d26f625060 msgr2=0x55d26f6296b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:46:59.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.269+0000 7f7f1c974700 1 --2- 172.21.15.32:0/3010490739 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d26f625060 0x55d26f6296b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:59.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.269+0000 7f7f1c974700 1 -- 172.21.15.32:0/3010490739 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d26f627740 con 0x55d26f624740 2024-05-03T18:46:59.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.269+0000 7f7f1d976700 1 --2- 172.21.15.32:0/3010490739 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d26f625060 0x55d26f6296b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T18:46:59.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.269+0000 7f7f1c974700 1 --2- 172.21.15.32:0/3010490739 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d26f624740 0x55d26f629170 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f7f18103e50 tx=0x7f7f08014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7b58b8c70e35054a server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:46:59.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.270+0000 7f7f02ffd700 1 -- 172.21.15.32:0/3010490739 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f08022070 con 0x55d26f624740 2024-05-03T18:46:59.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.270+0000 7f7f02ffd700 1 -- 172.21.15.32:0/3010490739 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f08004830 con 0x55d26f624740 2024-05-03T18:46:59.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.270+0000 7f7f02ffd700 1 -- 172.21.15.32:0/3010490739 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f080053e0 con 0x55d26f624740 2024-05-03T18:46:59.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.270+0000 7f7f29c6cb80 1 -- 172.21.15.32:0/3010490739 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d26f624740 msgr2=0x55d26f629170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:59.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.270+0000 7f7f29c6cb80 1 --2- 172.21.15.32:0/3010490739 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d26f624740 0x55d26f629170 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f7f18103e50 tx=0x7f7f08014000 comp rx=0 tx=0).stop 2024-05-03T18:46:59.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.270+0000 7f7f29c6cb80 1 -- 172.21.15.32:0/3010490739 shutdown_connections 2024-05-03T18:46:59.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.270+0000 7f7f29c6cb80 1 --2- 172.21.15.32:0/3010490739 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d26f625060 0x55d26f6296b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:59.273 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.270+0000 7f7f29c6cb80 1 --2- 172.21.15.32:0/3010490739 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d26f620480 0x55d26f628730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:59.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.270+0000 7f7f29c6cb80 1 --2- 172.21.15.32:0/3010490739 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d26f624740 0x55d26f629170 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:59.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.270+0000 7f7f29c6cb80 1 -- 172.21.15.32:0/3010490739 >> 172.21.15.32:0/3010490739 conn(0x55d26f54cb20 msgr2=0x55d26f61c040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:46:59.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.271+0000 7f7f29c6cb80 1 -- 172.21.15.32:0/3010490739 shutdown_connections 2024-05-03T18:46:59.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.271+0000 7f7f29c6cb80 1 -- 172.21.15.32:0/3010490739 wait complete. 2024-05-03T18:46:59.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.271+0000 7f7f29c6cb80 1 -- start start 2024-05-03T18:46:59.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.271+0000 7f7f29c6cb80 1 Processor -- start 2024-05-03T18:46:59.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.271+0000 7f7f29c6cb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d26f620480 0x55d26f61e280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:59.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.271+0000 7f7f29c6cb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d26f624740 0x55d26f61e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:59.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.271+0000 7f7f29c6cb80 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d26f625060 0x55d26f61ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:59.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.271+0000 7f7f29c6cb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55d26f63aca0 con 0x55d26f624740 2024-05-03T18:46:59.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.271+0000 7f7f29c6cb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55d26f63a5d0 con 0x55d26f620480 2024-05-03T18:46:59.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.271+0000 7f7f29c6cb80 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55d26f63b370 con 0x55d26f625060 2024-05-03T18:46:59.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.272+0000 7f7f29c6cb80 10 client.0 _ll_register_callbacks cb 0x55d26f506560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:46:59.277 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.272+0000 7f7f1d175700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d26f620480 0x55d26f61e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:59.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.272+0000 7f7f1d175700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d26f620480 0x55d26f61e280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.32:33332/0 (socket says 172.21.15.32:33332) 2024-05-03T18:46:59.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.272+0000 7f7f1d175700 1 -- 172.21.15.32:0/3259926205 learned_addr learned my addr 172.21.15.32:0/3259926205 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:46:59.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.272+0000 7f7f1c974700 1 --2- 172.21.15.32:0/3259926205 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d26f624740 0x55d26f61e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:59.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.272+0000 7f7f1d976700 1 --2- 172.21.15.32:0/3259926205 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d26f625060 0x55d26f61ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:59.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.272+0000 7f7f1d976700 1 -- 172.21.15.32:0/3259926205 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d26f620480 msgr2=0x55d26f61e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:46:59.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.272+0000 7f7f1d976700 1 --2- 172.21.15.32:0/3259926205 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d26f620480 0x55d26f61e280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:59.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.272+0000 7f7f1d976700 1 -- 172.21.15.32:0/3259926205 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d26f624740 msgr2=0x55d26f61e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:46:59.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.272+0000 7f7f1d976700 1 --2- 172.21.15.32:0/3259926205 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d26f624740 0x55d26f61e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:59.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.272+0000 7f7f1d976700 1 -- 172.21.15.32:0/3259926205 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d26f6406b0 con 0x55d26f625060 2024-05-03T18:46:59.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.272+0000 7f7f1d175700 1 --2- 172.21.15.32:0/3259926205 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d26f620480 0x55d26f61e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T18:46:59.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.272+0000 7f7f1c974700 1 --2- 172.21.15.32:0/3259926205 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d26f624740 0x55d26f61e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T18:46:59.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.272+0000 7f7f1d976700 1 --2- 172.21.15.32:0/3259926205 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d26f625060 0x55d26f61ed00 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7f10007f40 tx=0x7f7f10007e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c999e5c607141c84 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:46:59.278 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.272+0000 7f7f00ff9700 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-05-03T18:46:59.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.273+0000 7f7f00ff9700 1 -- 172.21.15.32:0/3259926205 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f10022040 con 0x55d26f625060 2024-05-03T18:46:59.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.273+0000 7f7f29c6cb80 1 -- 172.21.15.32:0/3259926205 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f08004830 con 0x55d26f625060 2024-05-03T18:46:59.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.273+0000 7f7f00ff9700 1 -- 172.21.15.32:0/3259926205 <== mon.1 v2:172.21.15.113:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8356+0+0 (secure 0 0 0) 0x7f7f10027450 con 0x55d26f625060 2024-05-03T18:46:59.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.273+0000 7f7f00ff9700 1 -- 172.21.15.32:0/3259926205 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7f1000b770 con 0x55d26f625060 2024-05-03T18:46:59.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.273+0000 7f7f29c6cb80 10 client.24724 fetch_fsmap learned FSMap version 25 2024-05-03T18:46:59.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.273+0000 7f7f29c6cb80 1 -- 172.21.15.32:0/3259926205 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d26f6a5130 con 0x55d26f625060 2024-05-03T18:46:59.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.273+0000 7f7f00ff9700 1 -- 172.21.15.32:0/3259926205 <== mon.1 v2:172.21.15.113:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f08004830 con 0x55d26f625060 2024-05-03T18:46:59.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.273+0000 7f7f00ff9700 1 -- 172.21.15.32:0/3259926205 <== mon.1 v2:172.21.15.113:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f7f10011070 con 0x55d26f625060 2024-05-03T18:46:59.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.273+0000 7f7f29c6cb80 10 client.24724 fetch_fsmap finished waiting for FSMap version 25 2024-05-03T18:46:59.279 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.273+0000 7f7f29c6cb80 10 client.24724 resolve_mds: resolved name 'f' to daemon mds.f 2024-05-03T18:46:59.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.273+0000 7f7f29c6cb80 20 client.24724 populate_metadata read hostname 'smithi032' 2024-05-03T18:46:59.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.273+0000 7f7f29c6cb80 1 --2- 172.21.15.32:0/3259926205 >> [v2:172.21.15.113:6834/2588738276,v1:172.21.15.113:6835/2588738276] conn(0x55d26f6a9780 0x55d26f6a8490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:46:59.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.273+0000 7f7f29c6cb80 4 client.24724 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-03T18:46:59.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.273+0000 7f7f29c6cb80 1 -- 172.21.15.32:0/3259926205 --> [v2:172.21.15.113:6834/2588738276,v1:172.21.15.113:6835/2588738276] -- command(tid 0: {"prefix": "status"}) -- 0x55d26f57a370 con 0x55d26f6a9780 2024-05-03T18:46:59.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.274+0000 7f7f1d175700 1 --2- 172.21.15.32:0/3259926205 >> [v2:172.21.15.113:6834/2588738276,v1:172.21.15.113:6835/2588738276] conn(0x55d26f6a9780 0x55d26f6a8490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:46:59.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.274+0000 7f7f00ff9700 10 client.24724 ms_handle_connect on v2:172.21.15.113:6834/2588738276 2024-05-03T18:46:59.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.274+0000 7f7f1d175700 1 --2- 172.21.15.32:0/3259926205 >> [v2:172.21.15.113:6834/2588738276,v1:172.21.15.113:6835/2588738276] conn(0x55d26f6a9780 0x55d26f6a8490 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=995a940f2df523d4 server_cookie=b1449d83da72d72e in_seq=0 out_seq=0 2024-05-03T18:46:59.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.274+0000 7f7f00ff9700 1 -- 172.21.15.32:0/3259926205 <== mds.2 v2:172.21.15.113:6834/2588738276 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55d26f57a370 con 0x55d26f6a9780 2024-05-03T18:46:59.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.274+0000 7f7f00ff9700 10 client.24724 handle_command_reply: tid=0 2024-05-03T18:46:59.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.274+0000 7f7f29c6cb80 1 client.24724 shutdown 2024-05-03T18:46:59.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.274+0000 7f7f29c6cb80 1 -- 172.21.15.32:0/3259926205 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d26f625060 msgr2=0x55d26f61ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:59.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.274+0000 7f7f29c6cb80 1 --2- 172.21.15.32:0/3259926205 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d26f625060 0x55d26f61ed00 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7f10007f40 tx=0x7f7f10007e80 comp rx=0 tx=0).stop 2024-05-03T18:46:59.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.274+0000 7f7f29c6cb80 1 -- 172.21.15.32:0/3259926205 shutdown_connections 2024-05-03T18:46:59.280 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.274+0000 7f7f29c6cb80 1 --2- 172.21.15.32:0/3259926205 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d26f625060 0x55d26f61ed00 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:59.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.274+0000 7f7f29c6cb80 1 --2- 172.21.15.32:0/3259926205 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d26f620480 0x55d26f61e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:59.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.274+0000 7f7f29c6cb80 1 --2- 172.21.15.32:0/3259926205 >> [v2:172.21.15.113:6834/2588738276,v1:172.21.15.113:6835/2588738276] conn(0x55d26f6a9780 0x55d26f6a8490 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:59.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.274+0000 7f7f29c6cb80 1 --2- 172.21.15.32:0/3259926205 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d26f624740 0x55d26f61e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:59.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.275+0000 7f7f00ff9700 0 client.24724 ms_handle_reset on v2:172.21.15.113:6834/2588738276 2024-05-03T18:46:59.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.275+0000 7f7f29c6cb80 1 -- 172.21.15.32:0/3259926205 >> 172.21.15.32:0/3259926205 conn(0x55d26f54cb20 msgr2=0x55d26f628730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:46:59.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.275+0000 7f7f29c6cb80 1 -- 172.21.15.32:0/3259926205 shutdown_connections 2024-05-03T18:46:59.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.275+0000 7f7f29c6cb80 1 -- 172.21.15.32:0/3259926205 wait complete. 2024-05-03T18:46:59.281 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.275+0000 7f7f29c6cb80 20 client.24724 trim_cache size 0 max 16384 2024-05-03T18:46:59.281 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-05-03T18:46:59.281 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_fsid": "f500455e-097b-11ef-bc97-c7b262605968", 2024-05-03T18:46:59.281 INFO:teuthology.orchestra.run.smithi032.stdout: "whoami": 2, 2024-05-03T18:46:59.281 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 24463, 2024-05-03T18:46:59.282 INFO:teuthology.orchestra.run.smithi032.stdout: "want_state": "up:active", 2024-05-03T18:46:59.282 INFO:teuthology.orchestra.run.smithi032.stdout: "state": "up:active", 2024-05-03T18:46:59.282 INFO:teuthology.orchestra.run.smithi032.stdout: "fs_name": "cephfs", 2024-05-03T18:46:59.282 INFO:teuthology.orchestra.run.smithi032.stdout: "rank_uptime": 115.802426262, 2024-05-03T18:46:59.282 INFO:teuthology.orchestra.run.smithi032.stdout: "mdsmap_epoch": 25, 2024-05-03T18:46:59.282 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch": 82, 2024-05-03T18:46:59.282 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch_barrier": 78, 2024-05-03T18:46:59.282 INFO:teuthology.orchestra.run.smithi032.stdout: "uptime": 159.70995955999999 2024-05-03T18:46:59.282 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-05-03T18:46:59.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.275+0000 7f7f1f3d9700 1 -- 172.21.15.32:0/2200040071 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f7f040b2990 msgr2=0x7f7f040b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:59.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.275+0000 7f7f1f3d9700 1 --2- 172.21.15.32:0/2200040071 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f7f040b2990 0x7f7f040b4e50 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f7f18079a40 tx=0x7f7f08009260 comp rx=0 tx=0).stop 2024-05-03T18:46:59.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.275+0000 7f7f1f3d9700 1 -- 172.21.15.32:0/2200040071 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f1806bc70 msgr2=0x7f7f18102410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:46:59.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.275+0000 7f7f1f3d9700 1 --2- 172.21.15.32:0/2200040071 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f1806bc70 0x7f7f18102410 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f7f1400d980 tx=0x7f7f1400dc90 comp rx=0 tx=0).stop 2024-05-03T18:46:59.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.275+0000 7f7f1f3d9700 1 -- 172.21.15.32:0/2200040071 shutdown_connections 2024-05-03T18:46:59.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.276+0000 7f7f1f3d9700 1 --2- 172.21.15.32:0/2200040071 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f7f040b2990 0x7f7f040b4e50 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:59.283 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.276+0000 7f7f1f3d9700 1 --2- 172.21.15.32:0/2200040071 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7f18109590 0x7f7f18102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:59.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.276+0000 7f7f1f3d9700 1 --2- 172.21.15.32:0/2200040071 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f1806bc70 0x7f7f18102410 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:59.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.276+0000 7f7f1f3d9700 1 --2- 172.21.15.32:0/2200040071 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7f1806c630 0x7f7f18102950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:46:59.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.276+0000 7f7f1f3d9700 1 -- 172.21.15.32:0/2200040071 >> 172.21.15.32:0/2200040071 conn(0x7f7f18079440 msgr2=0x7f7f18100f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:46:59.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.276+0000 7f7f1f3d9700 1 -- 172.21.15.32:0/2200040071 shutdown_connections 2024-05-03T18:46:59.284 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:46:59.276+0000 7f7f1f3d9700 1 -- 172.21.15.32:0/2200040071 wait complete. 2024-05-03T18:46:59.300 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:46:59 smithi113 ceph-mon[33540]: pgmap v348: 97 pgs: 97 active+clean; 631 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 100 B/s wr, 6 op/s; 59 B/s, 0 keys/s, 0 objects/s recovering 2024-05-03T18:46:59.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:46:59 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:46:59.331 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:46:59 smithi045 ceph-mon[33299]: pgmap v348: 97 pgs: 97 active+clean; 631 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 100 B/s wr, 6 op/s; 59 B/s, 0 keys/s, 0 objects/s recovering 2024-05-03T18:46:59.332 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:46:59 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:47:00.038 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid f500455e-097b-11ef-bc97-c7b262605968 -- ceph tell mds.c status 2024-05-03T18:47:00.788 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:47:01.200 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:47:00 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:47:01.201 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:47:00 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:47:01.201 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:47:00 smithi032 ceph-mon[29763]: pgmap v349: 97 pgs: 97 active+clean; 631 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 85 B/s wr, 5 op/s; 50 B/s, 0 keys/s, 0 objects/s recovering 2024-05-03T18:47:01.201 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:47:00 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:47:01.201 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:47:00 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:47:01.248 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:47:00 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:47:01.248 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:47:00 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:47:01.248 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:47:00 smithi045 ceph-mon[33299]: pgmap v349: 97 pgs: 97 active+clean; 631 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 85 B/s wr, 5 op/s; 50 B/s, 0 keys/s, 0 objects/s recovering 2024-05-03T18:47:01.248 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:47:00 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:47:01.248 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:47:00 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:47:01.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:47:00 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:47:01.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:47:00 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:47:01.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:47:00 smithi113 ceph-mon[33540]: pgmap v349: 97 pgs: 97 active+clean; 631 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 85 B/s wr, 5 op/s; 50 B/s, 0 keys/s, 0 objects/s recovering 2024-05-03T18:47:01.302 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:47:00 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:47:01.302 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:47:00 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:47:02.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.429+0000 7ff1a3435700 1 -- 172.21.15.32:0/2571042139 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff19c1095e0 msgr2=0x7ff19c1099c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:02.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.429+0000 7ff1a3435700 1 --2- 172.21.15.32:0/2571042139 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff19c1095e0 0x7ff19c1099c0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7ff1940099c0 tx=0x7ff194009cd0 comp rx=0 tx=0).stop 2024-05-03T18:47:02.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.429+0000 7ff1a3435700 1 -- 172.21.15.32:0/2571042139 shutdown_connections 2024-05-03T18:47:02.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.429+0000 7ff1a3435700 1 --2- 172.21.15.32:0/2571042139 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff19c103f00 0x7ff19c1103b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:02.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.429+0000 7ff1a3435700 1 --2- 172.21.15.32:0/2571042139 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff19c1095e0 0x7ff19c1099c0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:02.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.429+0000 7ff1a3435700 1 --2- 172.21.15.32:0/2571042139 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff19c103540 0x7ff19c1039c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:02.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.429+0000 7ff1a3435700 1 -- 172.21.15.32:0/2571042139 >> 172.21.15.32:0/2571042139 conn(0x7ff19c07a650 msgr2=0x7ff19c07aa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:47:02.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.429+0000 7ff1a3435700 1 -- 172.21.15.32:0/2571042139 shutdown_connections 2024-05-03T18:47:02.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.429+0000 7ff1a3435700 1 -- 172.21.15.32:0/2571042139 wait complete. 2024-05-03T18:47:02.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.430+0000 7ff1a3435700 1 Processor -- start 2024-05-03T18:47:02.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.430+0000 7ff1a3435700 1 -- start start 2024-05-03T18:47:02.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.430+0000 7ff1a3435700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff19c103540 0x7ff19c0757a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:02.433 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.430+0000 7ff1a3435700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff19c103f00 0x7ff19c075ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:02.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.430+0000 7ff1a3435700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff19c1095e0 0x7ff19c076220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:02.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.430+0000 7ff1a3435700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7ff19c113740 con 0x7ff19c103f00 2024-05-03T18:47:02.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.430+0000 7ff1a3435700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff19c1135c0 con 0x7ff19c103540 2024-05-03T18:47:02.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.430+0000 7ff1a3435700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7ff19c1138c0 con 0x7ff19c1095e0 2024-05-03T18:47:02.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.430+0000 7ff1a09d0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff19c103f00 0x7ff19c075ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:02.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.430+0000 7ff1a09d0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff19c103f00 0x7ff19c075ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58258/0 (socket says 172.21.15.32:58258) 2024-05-03T18:47:02.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.430+0000 7ff1a09d0700 1 -- 172.21.15.32:0/3327067290 learned_addr learned my addr 172.21.15.32:0/3327067290 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:47:02.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.431+0000 7ff1a19d2700 1 --2- 172.21.15.32:0/3327067290 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff19c1095e0 0x7ff19c076220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:02.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.431+0000 7ff1a11d1700 1 --2- 172.21.15.32:0/3327067290 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff19c103540 0x7ff19c0757a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:02.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.431+0000 7ff1a09d0700 1 -- 172.21.15.32:0/3327067290 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff19c1095e0 msgr2=0x7ff19c076220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:02.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.431+0000 7ff1a09d0700 1 --2- 172.21.15.32:0/3327067290 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff19c1095e0 0x7ff19c076220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:02.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.431+0000 7ff1a09d0700 1 -- 172.21.15.32:0/3327067290 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff19c103540 msgr2=0x7ff19c0757a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:02.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.431+0000 7ff1a09d0700 1 --2- 172.21.15.32:0/3327067290 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff19c103540 0x7ff19c0757a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:02.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.431+0000 7ff1a09d0700 1 -- 172.21.15.32:0/3327067290 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff19c1a80a0 con 0x7ff19c103f00 2024-05-03T18:47:02.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.431+0000 7ff1a19d2700 1 --2- 172.21.15.32:0/3327067290 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff19c1095e0 0x7ff19c076220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T18:47:02.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.431+0000 7ff1a11d1700 1 --2- 172.21.15.32:0/3327067290 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff19c103540 0x7ff19c0757a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T18:47:02.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.431+0000 7ff1a09d0700 1 --2- 172.21.15.32:0/3327067290 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff19c103f00 0x7ff19c075ce0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7ff19000d950 tx=0x7ff19000dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:47:02.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.431+0000 7ff19a7fc700 1 -- 172.21.15.32:0/3327067290 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff190016070 con 0x7ff19c103f00 2024-05-03T18:47:02.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.431+0000 7ff19a7fc700 1 -- 172.21.15.32:0/3327067290 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff190010460 con 0x7ff19c103f00 2024-05-03T18:47:02.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.431+0000 7ff1a3435700 1 -- 172.21.15.32:0/3327067290 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff19c1a8390 con 0x7ff19c103f00 2024-05-03T18:47:02.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.431+0000 7ff19a7fc700 1 -- 172.21.15.32:0/3327067290 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff19000f620 con 0x7ff19c103f00 2024-05-03T18:47:02.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.431+0000 7ff1a3435700 1 -- 172.21.15.32:0/3327067290 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff19c1a8920 con 0x7ff19c103f00 2024-05-03T18:47:02.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.433+0000 7ff19a7fc700 1 -- 172.21.15.32:0/3327067290 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7ff190019070 con 0x7ff19c103f00 2024-05-03T18:47:02.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.434+0000 7ff19a7fc700 1 --2- 172.21.15.32:0/3327067290 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7ff1840b2990 0x7ff1840b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:02.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.434+0000 7ff1a11d1700 1 --2- 172.21.15.32:0/3327067290 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7ff1840b2990 0x7ff1840b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:02.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.434+0000 7ff19a7fc700 1 -- 172.21.15.32:0/3327067290 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8356+0+0 (secure 0 0 0) 0x7ff1900e2100 con 0x7ff19c103f00 2024-05-03T18:47:02.437 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.435+0000 7ff1a11d1700 1 --2- 172.21.15.32:0/3327067290 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7ff1840b2990 0x7ff1840b4e50 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ff194003800 tx=0x7ff194005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:47:02.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.435+0000 7ff1adcc8b80 1 Processor -- start 2024-05-03T18:47:02.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.435+0000 7ff1adcc8b80 1 -- start start 2024-05-03T18:47:02.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.435+0000 7ff1adcc8b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557151ebfc10 0x557151ebfff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:02.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.435+0000 7ff1adcc8b80 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557151ec0530 0x557151ec3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:02.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.435+0000 7ff1adcc8b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557151ec4230 0x557151ec4690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:02.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.435+0000 7ff1adcc8b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x557151e12810 con 0x557151ebfc10 2024-05-03T18:47:02.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.435+0000 7ff1adcc8b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x557151db1770 con 0x557151ec4230 2024-05-03T18:47:02.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.435+0000 7ff1adcc8b80 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x557151e0e7d0 con 0x557151ec0530 2024-05-03T18:47:02.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.436+0000 7ff1a11d1700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557151ebfc10 0x557151ebfff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:02.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.436+0000 7ff1a09d0700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557151ec0530 0x557151ec3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:02.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.436+0000 7ff1a09d0700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557151ec0530 0x557151ec3cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.32:57932/0 (socket says 172.21.15.32:57932) 2024-05-03T18:47:02.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.436+0000 7ff1a11d1700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557151ebfc10 0x557151ebfff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58268/0 (socket says 172.21.15.32:58268) 2024-05-03T18:47:02.438 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.436+0000 7ff1a09d0700 1 -- 172.21.15.32:0/38235756 learned_addr learned my addr 172.21.15.32:0/38235756 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:47:02.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.436+0000 7ff1a19d2700 1 --2- 172.21.15.32:0/38235756 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557151ec4230 0x557151ec4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:02.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.436+0000 7ff1a11d1700 1 -- 172.21.15.32:0/38235756 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557151ec0530 msgr2=0x557151ec3cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:47:02.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.436+0000 7ff1a11d1700 1 --2- 172.21.15.32:0/38235756 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557151ec0530 0x557151ec3cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:02.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.436+0000 7ff1a11d1700 1 -- 172.21.15.32:0/38235756 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557151ec4230 msgr2=0x557151ec4690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:47:02.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.436+0000 7ff1a11d1700 1 --2- 172.21.15.32:0/38235756 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557151ec4230 0x557151ec4690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:02.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.436+0000 7ff1a11d1700 1 -- 172.21.15.32:0/38235756 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557151d5f880 con 0x557151ebfc10 2024-05-03T18:47:02.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.436+0000 7ff1a09d0700 1 --2- 172.21.15.32:0/38235756 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557151ec0530 0x557151ec3cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T18:47:02.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.436+0000 7ff1a11d1700 1 --2- 172.21.15.32:0/38235756 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557151ebfc10 0x557151ebfff0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ff194009990 tx=0x7ff19401f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a287d4677cf71053 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:47:02.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.436+0000 7ff1837fe700 1 -- 172.21.15.32:0/38235756 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff194003860 con 0x557151ebfc10 2024-05-03T18:47:02.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.436+0000 7ff1837fe700 1 -- 172.21.15.32:0/38235756 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff194005030 con 0x557151ebfc10 2024-05-03T18:47:02.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.436+0000 7ff1837fe700 1 -- 172.21.15.32:0/38235756 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff19402f6c0 con 0x557151ebfc10 2024-05-03T18:47:02.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.437+0000 7ff1adcc8b80 1 -- 172.21.15.32:0/38235756 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557151ebfc10 msgr2=0x557151ebfff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:02.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.437+0000 7ff1adcc8b80 1 --2- 172.21.15.32:0/38235756 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557151ebfc10 0x557151ebfff0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ff194009990 tx=0x7ff19401f000 comp rx=0 tx=0).stop 2024-05-03T18:47:02.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.437+0000 7ff1adcc8b80 1 -- 172.21.15.32:0/38235756 shutdown_connections 2024-05-03T18:47:02.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.437+0000 7ff1adcc8b80 1 --2- 172.21.15.32:0/38235756 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557151ec4230 0x557151ec4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:02.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.437+0000 7ff1adcc8b80 1 --2- 172.21.15.32:0/38235756 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557151ebfc10 0x557151ebfff0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:02.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.437+0000 7ff1adcc8b80 1 --2- 172.21.15.32:0/38235756 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557151ec0530 0x557151ec3cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:02.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.437+0000 7ff1adcc8b80 1 -- 172.21.15.32:0/38235756 >> 172.21.15.32:0/38235756 conn(0x557151dec0b0 msgr2=0x557151eba280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:47:02.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.438+0000 7ff1adcc8b80 1 -- 172.21.15.32:0/38235756 shutdown_connections 2024-05-03T18:47:02.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.438+0000 7ff1adcc8b80 1 -- 172.21.15.32:0/38235756 wait complete. 2024-05-03T18:47:02.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.438+0000 7ff1adcc8b80 1 -- start start 2024-05-03T18:47:02.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.438+0000 7ff1adcc8b80 1 Processor -- start 2024-05-03T18:47:02.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.438+0000 7ff1adcc8b80 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557151ebfc10 0x557151ecb690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:02.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.438+0000 7ff1adcc8b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557151ec0530 0x557151ec8a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:02.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.438+0000 7ff1adcc8b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557151ec4230 0x557151ec8f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:02.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.439+0000 7ff1adcc8b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x557151ecc430 con 0x557151ec0530 2024-05-03T18:47:02.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.439+0000 7ff1adcc8b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x557151d78be0 con 0x557151ec4230 2024-05-03T18:47:02.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.439+0000 7ff1adcc8b80 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x557151ecc5b0 con 0x557151ebfc10 2024-05-03T18:47:02.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.439+0000 7ff1adcc8b80 10 client.0 _ll_register_callbacks cb 0x557151daa8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:47:02.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.439+0000 7ff1a09d0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557151ec0530 0x557151ec8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:02.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.439+0000 7ff1a11d1700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557151ebfc10 0x557151ecb690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:02.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.439+0000 7ff1a09d0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557151ec0530 0x557151ec8a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58284/0 (socket says 172.21.15.32:58284) 2024-05-03T18:47:02.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.439+0000 7ff1a09d0700 1 -- 172.21.15.32:0/4099291788 learned_addr learned my addr 172.21.15.32:0/4099291788 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:47:02.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.439+0000 7ff1a19d2700 1 --2- 172.21.15.32:0/4099291788 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557151ec4230 0x557151ec8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:02.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.439+0000 7ff1a11d1700 1 -- 172.21.15.32:0/4099291788 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557151ec4230 msgr2=0x557151ec8f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:47:02.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.439+0000 7ff1a11d1700 1 --2- 172.21.15.32:0/4099291788 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557151ec4230 0x557151ec8f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:02.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.439+0000 7ff1a11d1700 1 -- 172.21.15.32:0/4099291788 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557151ec0530 msgr2=0x557151ec8a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:47:02.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.439+0000 7ff1a11d1700 1 --2- 172.21.15.32:0/4099291788 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557151ec0530 0x557151ec8a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:02.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.439+0000 7ff1a11d1700 1 -- 172.21.15.32:0/4099291788 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557151dae640 con 0x557151ebfc10 2024-05-03T18:47:02.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.439+0000 7ff1a09d0700 1 --2- 172.21.15.32:0/4099291788 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557151ec0530 0x557151ec8a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T18:47:02.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.439+0000 7ff1a11d1700 1 --2- 172.21.15.32:0/4099291788 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557151ebfc10 0x557151ecb690 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff194000c00 tx=0x7ff194005810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c51ce998c5e7ea42 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:47:02.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.439+0000 7ff180ff9700 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-05-03T18:47:02.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.440+0000 7ff180ff9700 1 -- 172.21.15.32:0/4099291788 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff19403c060 con 0x557151ebfc10 2024-05-03T18:47:02.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.440+0000 7ff1adcc8b80 1 -- 172.21.15.32:0/4099291788 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff194005030 con 0x557151ebfc10 2024-05-03T18:47:02.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.440+0000 7ff180ff9700 1 -- 172.21.15.32:0/4099291788 <== mon.1 v2:172.21.15.113:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8356+0+0 (secure 0 0 0) 0x7ff194003bf0 con 0x557151ebfc10 2024-05-03T18:47:02.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.440+0000 7ff180ff9700 1 -- 172.21.15.32:0/4099291788 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff19402f460 con 0x557151ebfc10 2024-05-03T18:47:02.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.440+0000 7ff1adcc8b80 10 client.24733 fetch_fsmap learned FSMap version 25 2024-05-03T18:47:02.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.440+0000 7ff1adcc8b80 1 -- 172.21.15.32:0/4099291788 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x557151ecc430 con 0x557151ebfc10 2024-05-03T18:47:02.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.440+0000 7ff180ff9700 1 -- 172.21.15.32:0/4099291788 <== mon.1 v2:172.21.15.113:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff194005030 con 0x557151ebfc10 2024-05-03T18:47:02.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.440+0000 7ff180ff9700 1 -- 172.21.15.32:0/4099291788 <== mon.1 v2:172.21.15.113:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7ff19400f6a0 con 0x557151ebfc10 2024-05-03T18:47:02.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.440+0000 7ff1adcc8b80 10 client.24733 fetch_fsmap finished waiting for FSMap version 25 2024-05-03T18:47:02.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.440+0000 7ff1adcc8b80 10 client.24733 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-03T18:47:02.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.440+0000 7ff1adcc8b80 20 client.24733 populate_metadata read hostname 'smithi032' 2024-05-03T18:47:02.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.440+0000 7ff1adcc8b80 1 --2- 172.21.15.32:0/4099291788 >> [v2:172.21.15.113:6836/3677284460,v1:172.21.15.113:6837/3677284460] conn(0x557151ed8cf0 0x557151eda160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:02.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.440+0000 7ff1adcc8b80 4 client.24733 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-03T18:47:02.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.440+0000 7ff1adcc8b80 1 -- 172.21.15.32:0/4099291788 --> [v2:172.21.15.113:6836/3677284460,v1:172.21.15.113:6837/3677284460] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557151e201f0 con 0x557151ed8cf0 2024-05-03T18:47:02.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.441+0000 7ff1a09d0700 1 --2- 172.21.15.32:0/4099291788 >> [v2:172.21.15.113:6836/3677284460,v1:172.21.15.113:6837/3677284460] conn(0x557151ed8cf0 0x557151eda160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:02.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.441+0000 7ff1a09d0700 1 --2- 172.21.15.32:0/4099291788 >> [v2:172.21.15.113:6836/3677284460,v1:172.21.15.113:6837/3677284460] conn(0x557151ed8cf0 0x557151eda160 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e875b7ea6f27e9de server_cookie=2b2aba9657d5a295 in_seq=0 out_seq=0 2024-05-03T18:47:02.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.441+0000 7ff180ff9700 10 client.24733 ms_handle_connect on v2:172.21.15.113:6836/3677284460 2024-05-03T18:47:02.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.442+0000 7ff180ff9700 1 -- 172.21.15.32:0/4099291788 <== mds.1 v2:172.21.15.113:6836/3677284460 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x557151e201f0 con 0x557151ed8cf0 2024-05-03T18:47:02.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.442+0000 7ff180ff9700 10 client.24733 handle_command_reply: tid=0 2024-05-03T18:47:02.446 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.442+0000 7ff1adcc8b80 1 client.24733 shutdown 2024-05-03T18:47:02.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.442+0000 7ff1adcc8b80 1 -- 172.21.15.32:0/4099291788 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557151ebfc10 msgr2=0x557151ecb690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:02.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.442+0000 7ff1adcc8b80 1 --2- 172.21.15.32:0/4099291788 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557151ebfc10 0x557151ecb690 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff194000c00 tx=0x7ff194005810 comp rx=0 tx=0).stop 2024-05-03T18:47:02.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.442+0000 7ff1adcc8b80 1 -- 172.21.15.32:0/4099291788 shutdown_connections 2024-05-03T18:47:02.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.442+0000 7ff1adcc8b80 1 --2- 172.21.15.32:0/4099291788 >> [v2:172.21.15.113:6836/3677284460,v1:172.21.15.113:6837/3677284460] conn(0x557151ed8cf0 0x557151eda160 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:02.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.442+0000 7ff1adcc8b80 1 --2- 172.21.15.32:0/4099291788 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557151ec4230 0x557151ec8f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:02.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.442+0000 7ff1adcc8b80 1 --2- 172.21.15.32:0/4099291788 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557151ebfc10 0x557151ecb690 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:02.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.442+0000 7ff1adcc8b80 1 --2- 172.21.15.32:0/4099291788 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557151ec0530 0x557151ec8a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:02.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.442+0000 7ff180ff9700 0 client.24733 ms_handle_reset on v2:172.21.15.113:6836/3677284460 2024-05-03T18:47:02.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.442+0000 7ff1adcc8b80 1 -- 172.21.15.32:0/4099291788 >> 172.21.15.32:0/4099291788 conn(0x557151dec0b0 msgr2=0x557151ec56e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:47:02.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.442+0000 7ff1adcc8b80 1 -- 172.21.15.32:0/4099291788 shutdown_connections 2024-05-03T18:47:02.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.442+0000 7ff1adcc8b80 1 -- 172.21.15.32:0/4099291788 wait complete. 2024-05-03T18:47:02.447 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.443+0000 7ff1adcc8b80 20 client.24733 trim_cache size 0 max 16384 2024-05-03T18:47:02.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.468+0000 7ff1adcc8b80 1 Processor -- start 2024-05-03T18:47:02.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.468+0000 7ff1adcc8b80 1 -- start start 2024-05-03T18:47:02.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.469+0000 7ff1adcc8b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557151ebfc10 0x557151ec8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:02.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.469+0000 7ff1adcc8b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557151ec0530 0x557151ecaed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:02.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.469+0000 7ff1adcc8b80 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557151ec4230 0x557151ecb410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:02.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.469+0000 7ff1adcc8b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x557151ed84a0 con 0x557151ec0530 2024-05-03T18:47:02.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.469+0000 7ff1adcc8b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x557151ed8b70 con 0x557151ebfc10 2024-05-03T18:47:02.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.469+0000 7ff1adcc8b80 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x557151ed9240 con 0x557151ec4230 2024-05-03T18:47:02.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.469+0000 7ff1a09d0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557151ec0530 0x557151ecaed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:02.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.469+0000 7ff1a09d0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557151ec0530 0x557151ecaed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58298/0 (socket says 172.21.15.32:58298) 2024-05-03T18:47:02.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.469+0000 7ff1a09d0700 1 -- 172.21.15.32:0/2525290401 learned_addr learned my addr 172.21.15.32:0/2525290401 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:47:02.472 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.469+0000 7ff1a11d1700 1 --2- 172.21.15.32:0/2525290401 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557151ebfc10 0x557151ec8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:02.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.469+0000 7ff1a19d2700 1 --2- 172.21.15.32:0/2525290401 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557151ec4230 0x557151ecb410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:02.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.469+0000 7ff1a09d0700 1 -- 172.21.15.32:0/2525290401 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557151ec4230 msgr2=0x557151ecb410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:47:02.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.469+0000 7ff1a09d0700 1 --2- 172.21.15.32:0/2525290401 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557151ec4230 0x557151ecb410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:02.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.469+0000 7ff1a09d0700 1 -- 172.21.15.32:0/2525290401 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557151ebfc10 msgr2=0x557151ec8fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:47:02.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.469+0000 7ff1a09d0700 1 --2- 172.21.15.32:0/2525290401 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557151ebfc10 0x557151ec8fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:02.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.469+0000 7ff1a09d0700 1 -- 172.21.15.32:0/2525290401 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557151eb51f0 con 0x557151ec0530 2024-05-03T18:47:02.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.469+0000 7ff1a09d0700 1 --2- 172.21.15.32:0/2525290401 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557151ec0530 0x557151ecaed0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7ff1900a91e0 tx=0x7ff190091040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d2046906f72c3fc server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:47:02.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.470+0000 7ff182ffd700 1 -- 172.21.15.32:0/2525290401 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1900c3070 con 0x557151ec0530 2024-05-03T18:47:02.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.470+0000 7ff182ffd700 1 -- 172.21.15.32:0/2525290401 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1900910b0 con 0x557151ec0530 2024-05-03T18:47:02.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.470+0000 7ff182ffd700 1 -- 172.21.15.32:0/2525290401 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff190014030 con 0x557151ec0530 2024-05-03T18:47:02.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.471+0000 7ff1adcc8b80 1 -- 172.21.15.32:0/2525290401 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557151ec0530 msgr2=0x557151ecaed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:02.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.471+0000 7ff1adcc8b80 1 --2- 172.21.15.32:0/2525290401 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557151ec0530 0x557151ecaed0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7ff1900a91e0 tx=0x7ff190091040 comp rx=0 tx=0).stop 2024-05-03T18:47:02.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.471+0000 7ff1adcc8b80 1 -- 172.21.15.32:0/2525290401 shutdown_connections 2024-05-03T18:47:02.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.471+0000 7ff1adcc8b80 1 --2- 172.21.15.32:0/2525290401 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557151ec4230 0x557151ecb410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:02.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.471+0000 7ff1adcc8b80 1 --2- 172.21.15.32:0/2525290401 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557151ebfc10 0x557151ec8fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:02.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.471+0000 7ff1adcc8b80 1 --2- 172.21.15.32:0/2525290401 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557151ec0530 0x557151ecaed0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:02.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.471+0000 7ff1adcc8b80 1 -- 172.21.15.32:0/2525290401 >> 172.21.15.32:0/2525290401 conn(0x557151dec0b0 msgr2=0x557151ec7320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:47:02.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.471+0000 7ff1adcc8b80 1 -- 172.21.15.32:0/2525290401 shutdown_connections 2024-05-03T18:47:02.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.471+0000 7ff1adcc8b80 1 -- 172.21.15.32:0/2525290401 wait complete. 2024-05-03T18:47:02.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.471+0000 7ff1adcc8b80 1 -- start start 2024-05-03T18:47:02.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.471+0000 7ff1adcc8b80 1 Processor -- start 2024-05-03T18:47:02.474 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.472+0000 7ff1adcc8b80 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557151ebfc10 0x557151ec1410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:02.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.472+0000 7ff1adcc8b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557151ec0530 0x557151ec1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:02.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.472+0000 7ff1adcc8b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557151ec4230 0x557151ee25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:02.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.472+0000 7ff1adcc8b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x557151ed9fe0 con 0x557151ec4230 2024-05-03T18:47:02.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.472+0000 7ff1adcc8b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x557151ed9910 con 0x557151ec0530 2024-05-03T18:47:02.477 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.472+0000 7ff1adcc8b80 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x557151eda6b0 con 0x557151ebfc10 2024-05-03T18:47:02.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.472+0000 7ff1a11d1700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557151ebfc10 0x557151ec1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:02.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.472+0000 7ff1a11d1700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557151ebfc10 0x557151ec1410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.32:57968/0 (socket says 172.21.15.32:57968) 2024-05-03T18:47:02.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.472+0000 7ff1a11d1700 1 -- 172.21.15.32:0/1705944706 learned_addr learned my addr 172.21.15.32:0/1705944706 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:47:02.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.472+0000 7ff1adcc8b80 10 client.0 _ll_register_callbacks cb 0x557151daa8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:47:02.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.472+0000 7ff1a19d2700 1 --2- 172.21.15.32:0/1705944706 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557151ec4230 0x557151ee25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:02.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.472+0000 7ff1a11d1700 1 -- 172.21.15.32:0/1705944706 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557151ec0530 msgr2=0x557151ec1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:47:02.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.472+0000 7ff1a09d0700 1 --2- 172.21.15.32:0/1705944706 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557151ec0530 0x557151ec1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:02.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.472+0000 7ff1a11d1700 1 --2- 172.21.15.32:0/1705944706 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557151ec0530 0x557151ec1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:02.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.472+0000 7ff1a11d1700 1 -- 172.21.15.32:0/1705944706 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557151ec4230 msgr2=0x557151ee25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:47:02.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.472+0000 7ff1a11d1700 1 --2- 172.21.15.32:0/1705944706 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557151ec4230 0x557151ee25d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:02.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.472+0000 7ff1a11d1700 1 -- 172.21.15.32:0/1705944706 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557151ee2bc0 con 0x557151ebfc10 2024-05-03T18:47:02.478 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.472+0000 7ff1a19d2700 1 --2- 172.21.15.32:0/1705944706 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557151ec4230 0x557151ee25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T18:47:02.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.472+0000 7ff1a09d0700 1 --2- 172.21.15.32:0/1705944706 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557151ec0530 0x557151ec1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:47:02.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.472+0000 7ff1a11d1700 1 --2- 172.21.15.32:0/1705944706 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557151ebfc10 0x557151ec1410 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff194009350 tx=0x7ff19400be40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=788116e3e4090dc2 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:47:02.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.472+0000 7ff180ff9700 10 client.0 ms_handle_connect on v2:172.21.15.113:3300/0 2024-05-03T18:47:02.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.473+0000 7ff180ff9700 1 -- 172.21.15.32:0/1705944706 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff19403c070 con 0x557151ebfc10 2024-05-03T18:47:02.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.473+0000 7ff180ff9700 1 -- 172.21.15.32:0/1705944706 <== mon.1 v2:172.21.15.113:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8356+0+0 (secure 0 0 0) 0x7ff194003cf0 con 0x557151ebfc10 2024-05-03T18:47:02.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.473+0000 7ff1adcc8b80 1 -- 172.21.15.32:0/1705944706 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff1900910b0 con 0x557151ebfc10 2024-05-03T18:47:02.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.473+0000 7ff180ff9700 1 -- 172.21.15.32:0/1705944706 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff194050050 con 0x557151ebfc10 2024-05-03T18:47:02.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.473+0000 7ff1adcc8b80 10 client.24739 fetch_fsmap learned FSMap version 25 2024-05-03T18:47:02.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.473+0000 7ff1adcc8b80 1 -- 172.21.15.32:0/1705944706 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x557151ee3e80 con 0x557151ebfc10 2024-05-03T18:47:02.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.473+0000 7ff180ff9700 1 -- 172.21.15.32:0/1705944706 <== mon.1 v2:172.21.15.113:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1900910b0 con 0x557151ebfc10 2024-05-03T18:47:02.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.473+0000 7ff180ff9700 1 -- 172.21.15.32:0/1705944706 <== mon.1 v2:172.21.15.113:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7ff194048020 con 0x557151ebfc10 2024-05-03T18:47:02.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.474+0000 7ff1adcc8b80 10 client.24739 fetch_fsmap finished waiting for FSMap version 25 2024-05-03T18:47:02.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.474+0000 7ff1adcc8b80 10 client.24739 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-03T18:47:02.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.474+0000 7ff1adcc8b80 20 client.24739 populate_metadata read hostname 'smithi032' 2024-05-03T18:47:02.479 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.474+0000 7ff1adcc8b80 1 --2- 172.21.15.32:0/1705944706 >> [v2:172.21.15.113:6836/3677284460,v1:172.21.15.113:6837/3677284460] conn(0x557151f489a0 0x557151f476b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:02.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.474+0000 7ff1adcc8b80 4 client.24739 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-03T18:47:02.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.474+0000 7ff1adcc8b80 1 -- 172.21.15.32:0/1705944706 --> [v2:172.21.15.113:6836/3677284460,v1:172.21.15.113:6837/3677284460] -- command(tid 0: {"prefix": "status"}) -- 0x557151e1a290 con 0x557151f489a0 2024-05-03T18:47:02.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.474+0000 7ff1a09d0700 1 --2- 172.21.15.32:0/1705944706 >> [v2:172.21.15.113:6836/3677284460,v1:172.21.15.113:6837/3677284460] conn(0x557151f489a0 0x557151f476b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:02.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.475+0000 7ff1a09d0700 1 --2- 172.21.15.32:0/1705944706 >> [v2:172.21.15.113:6836/3677284460,v1:172.21.15.113:6837/3677284460] conn(0x557151f489a0 0x557151f476b0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4a4cf0b3d96b02d8 server_cookie=1f8071bdd5557ce3 in_seq=0 out_seq=0 2024-05-03T18:47:02.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.475+0000 7ff180ff9700 10 client.24739 ms_handle_connect on v2:172.21.15.113:6836/3677284460 2024-05-03T18:47:02.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.475+0000 7ff180ff9700 1 -- 172.21.15.32:0/1705944706 <== mds.1 v2:172.21.15.113:6836/3677284460 1 ==== command_reply(tid 0: 0 ) ==== 8+0+321 (crc 0 0 0) 0x557151e1a290 con 0x557151f489a0 2024-05-03T18:47:02.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.475+0000 7ff180ff9700 10 client.24739 handle_command_reply: tid=0 2024-05-03T18:47:02.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.475+0000 7ff1adcc8b80 1 client.24739 shutdown 2024-05-03T18:47:02.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.475+0000 7ff1adcc8b80 1 -- 172.21.15.32:0/1705944706 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557151ebfc10 msgr2=0x557151ec1410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:02.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.475+0000 7ff1adcc8b80 1 --2- 172.21.15.32:0/1705944706 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557151ebfc10 0x557151ec1410 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff194009350 tx=0x7ff19400be40 comp rx=0 tx=0).stop 2024-05-03T18:47:02.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.475+0000 7ff1adcc8b80 1 -- 172.21.15.32:0/1705944706 shutdown_connections 2024-05-03T18:47:02.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.475+0000 7ff1adcc8b80 1 --2- 172.21.15.32:0/1705944706 >> [v2:172.21.15.113:6836/3677284460,v1:172.21.15.113:6837/3677284460] conn(0x557151f489a0 0x557151f476b0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:02.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.475+0000 7ff1adcc8b80 1 --2- 172.21.15.32:0/1705944706 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557151ec4230 0x557151ee25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:02.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.475+0000 7ff1adcc8b80 1 --2- 172.21.15.32:0/1705944706 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557151ebfc10 0x557151ec1410 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:02.480 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.475+0000 7ff1adcc8b80 1 --2- 172.21.15.32:0/1705944706 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557151ec0530 0x557151ec1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:02.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.475+0000 7ff1adcc8b80 1 -- 172.21.15.32:0/1705944706 >> 172.21.15.32:0/1705944706 conn(0x557151dec0b0 msgr2=0x557151ebf2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:47:02.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.475+0000 7ff180ff9700 0 client.24739 ms_handle_reset on v2:172.21.15.113:6836/3677284460 2024-05-03T18:47:02.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.476+0000 7ff1adcc8b80 1 -- 172.21.15.32:0/1705944706 shutdown_connections 2024-05-03T18:47:02.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.476+0000 7ff1adcc8b80 1 -- 172.21.15.32:0/1705944706 wait complete. 2024-05-03T18:47:02.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.476+0000 7ff1adcc8b80 20 client.24739 trim_cache size 0 max 16384 2024-05-03T18:47:02.481 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-05-03T18:47:02.481 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_fsid": "f500455e-097b-11ef-bc97-c7b262605968", 2024-05-03T18:47:02.481 INFO:teuthology.orchestra.run.smithi032.stdout: "whoami": 1, 2024-05-03T18:47:02.482 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 24469, 2024-05-03T18:47:02.482 INFO:teuthology.orchestra.run.smithi032.stdout: "want_state": "up:active", 2024-05-03T18:47:02.482 INFO:teuthology.orchestra.run.smithi032.stdout: "state": "up:active", 2024-05-03T18:47:02.482 INFO:teuthology.orchestra.run.smithi032.stdout: "fs_name": "cephfs", 2024-05-03T18:47:02.482 INFO:teuthology.orchestra.run.smithi032.stdout: "rank_uptime": 120.008684683, 2024-05-03T18:47:02.482 INFO:teuthology.orchestra.run.smithi032.stdout: "mdsmap_epoch": 25, 2024-05-03T18:47:02.482 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch": 82, 2024-05-03T18:47:02.482 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch_barrier": 78, 2024-05-03T18:47:02.482 INFO:teuthology.orchestra.run.smithi032.stdout: "uptime": 160.16529745 2024-05-03T18:47:02.482 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-05-03T18:47:02.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.476+0000 7ff1a3435700 1 -- 172.21.15.32:0/3327067290 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7ff1840b2990 msgr2=0x7ff1840b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:02.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.476+0000 7ff1a3435700 1 --2- 172.21.15.32:0/3327067290 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7ff1840b2990 0x7ff1840b4e50 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ff194003800 tx=0x7ff194005fd0 comp rx=0 tx=0).stop 2024-05-03T18:47:02.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.476+0000 7ff1a3435700 1 -- 172.21.15.32:0/3327067290 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff19c103f00 msgr2=0x7ff19c075ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:02.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.476+0000 7ff1a3435700 1 --2- 172.21.15.32:0/3327067290 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff19c103f00 0x7ff19c075ce0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7ff19000d950 tx=0x7ff19000dd10 comp rx=0 tx=0).stop 2024-05-03T18:47:02.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.476+0000 7ff1a3435700 1 -- 172.21.15.32:0/3327067290 shutdown_connections 2024-05-03T18:47:02.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.476+0000 7ff1a3435700 1 --2- 172.21.15.32:0/3327067290 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7ff1840b2990 0x7ff1840b4e50 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:02.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.476+0000 7ff1a3435700 1 --2- 172.21.15.32:0/3327067290 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff19c1095e0 0x7ff19c076220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:02.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.476+0000 7ff1a3435700 1 --2- 172.21.15.32:0/3327067290 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff19c103540 0x7ff19c0757a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:02.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.476+0000 7ff1a3435700 1 --2- 172.21.15.32:0/3327067290 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff19c103f00 0x7ff19c075ce0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:02.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.476+0000 7ff1a3435700 1 -- 172.21.15.32:0/3327067290 >> 172.21.15.32:0/3327067290 conn(0x7ff19c07a650 msgr2=0x7ff19c102170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:47:02.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.477+0000 7ff1a3435700 1 -- 172.21.15.32:0/3327067290 shutdown_connections 2024-05-03T18:47:02.483 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:02.477+0000 7ff1a3435700 1 -- 172.21.15.32:0/3327067290 wait complete. 2024-05-03T18:47:03.060 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid f500455e-097b-11ef-bc97-c7b262605968 -- ceph tell mds.i status 2024-05-03T18:47:03.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:47:03 smithi032 ceph-mon[29763]: pgmap v350: 97 pgs: 97 active+clean; 631 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 85 B/s wr, 5 op/s; 50 B/s, 0 objects/s recovering 2024-05-03T18:47:03.336 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:47:03.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:47:03 smithi045 ceph-mon[33299]: pgmap v350: 97 pgs: 97 active+clean; 631 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 85 B/s wr, 5 op/s; 50 B/s, 0 objects/s recovering 2024-05-03T18:47:03.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:47:03 smithi113 ceph-mon[33540]: pgmap v350: 97 pgs: 97 active+clean; 631 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 85 B/s wr, 5 op/s; 50 B/s, 0 objects/s recovering 2024-05-03T18:47:04.462 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:47:04 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:47:04.462 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:47:04 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:47:04.462 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:47:04 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:47:04.462 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:47:04 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:47:04.462 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:47:04 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:47:04.497 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:47:04 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:47:04.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:47:04 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:47:04.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:47:04 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:47:04.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:47:04 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:47:04.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:47:04 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:47:04.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:47:04 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:47:04.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:47:04 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:47:04.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:47:04 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:47:04.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:47:04 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:47:04.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:47:04 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:47:04.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:04.984+0000 7fd97131c700 1 -- 172.21.15.32:0/396425069 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd96c06ba10 msgr2=0x7fd96c06be90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:04.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:04.984+0000 7fd97131c700 1 --2- 172.21.15.32:0/396425069 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd96c06ba10 0x7fd96c06be90 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fd960009990 tx=0x7fd960009ca0 comp rx=0 tx=0).stop 2024-05-03T18:47:04.985 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:04.984+0000 7fd97131c700 1 -- 172.21.15.32:0/396425069 shutdown_connections 2024-05-03T18:47:04.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:04.984+0000 7fd97131c700 1 --2- 172.21.15.32:0/396425069 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd96c06c3d0 0x7fd96c110320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:04.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:04.984+0000 7fd97131c700 1 --2- 172.21.15.32:0/396425069 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd96c109500 0x7fd96c1098e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:04.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:04.984+0000 7fd97131c700 1 --2- 172.21.15.32:0/396425069 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd96c06ba10 0x7fd96c06be90 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:04.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:04.984+0000 7fd97131c700 1 -- 172.21.15.32:0/396425069 >> 172.21.15.32:0/396425069 conn(0x7fd96c079310 msgr2=0x7fd96c079720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:47:04.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:04.984+0000 7fd97131c700 1 -- 172.21.15.32:0/396425069 shutdown_connections 2024-05-03T18:47:04.986 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:04.984+0000 7fd97131c700 1 -- 172.21.15.32:0/396425069 wait complete. 2024-05-03T18:47:04.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:04.985+0000 7fd97131c700 1 Processor -- start 2024-05-03T18:47:04.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:04.985+0000 7fd97131c700 1 -- start start 2024-05-03T18:47:04.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:04.985+0000 7fd97131c700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd96c06ba10 0x7fd96c19f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:04.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:04.985+0000 7fd97131c700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd96c06c3d0 0x7fd96c19fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:04.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:04.985+0000 7fd97131c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd96c109500 0x7fd96c199990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:04.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:04.985+0000 7fd97131c700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fd96c113690 con 0x7fd96c109500 2024-05-03T18:47:04.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:04.985+0000 7fd97131c700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd96c113510 con 0x7fd96c06ba10 2024-05-03T18:47:04.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:04.985+0000 7fd97131c700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fd96c113810 con 0x7fd96c06c3d0 2024-05-03T18:47:04.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:04.985+0000 7fd96b7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd96c109500 0x7fd96c199990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:04.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:04.985+0000 7fd96b7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd96c109500 0x7fd96c199990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57448/0 (socket says 172.21.15.32:57448) 2024-05-03T18:47:04.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:04.985+0000 7fd96b7fe700 1 -- 172.21.15.32:0/1023754064 learned_addr learned my addr 172.21.15.32:0/1023754064 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:47:04.988 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:04.985+0000 7fd96affd700 1 --2- 172.21.15.32:0/1023754064 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd96c06ba10 0x7fd96c19f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:04.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:04.986+0000 7fd96a7fc700 1 --2- 172.21.15.32:0/1023754064 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd96c06c3d0 0x7fd96c19fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:04.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:04.986+0000 7fd96b7fe700 1 -- 172.21.15.32:0/1023754064 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd96c06c3d0 msgr2=0x7fd96c19fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:04.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:04.986+0000 7fd96b7fe700 1 --2- 172.21.15.32:0/1023754064 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd96c06c3d0 0x7fd96c19fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:04.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:04.986+0000 7fd96b7fe700 1 -- 172.21.15.32:0/1023754064 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd96c06ba10 msgr2=0x7fd96c19f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:04.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:04.986+0000 7fd96b7fe700 1 --2- 172.21.15.32:0/1023754064 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd96c06ba10 0x7fd96c19f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:04.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:04.986+0000 7fd96b7fe700 1 -- 172.21.15.32:0/1023754064 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd96c19a0f0 con 0x7fd96c109500 2024-05-03T18:47:04.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:04.986+0000 7fd96b7fe700 1 --2- 172.21.15.32:0/1023754064 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd96c109500 0x7fd96c199990 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fd96400c900 tx=0x7fd96400cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:47:04.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:04.986+0000 7fd953fff700 1 -- 172.21.15.32:0/1023754064 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd964007b00 con 0x7fd96c109500 2024-05-03T18:47:04.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:04.986+0000 7fd97131c700 1 -- 172.21.15.32:0/1023754064 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd96c19a3e0 con 0x7fd96c109500 2024-05-03T18:47:04.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:04.986+0000 7fd953fff700 1 -- 172.21.15.32:0/1023754064 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd96400ce50 con 0x7fd96c109500 2024-05-03T18:47:04.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:04.986+0000 7fd953fff700 1 -- 172.21.15.32:0/1023754064 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd964018660 con 0x7fd96c109500 2024-05-03T18:47:04.989 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:04.986+0000 7fd97131c700 1 -- 172.21.15.32:0/1023754064 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd96c1a66a0 con 0x7fd96c109500 2024-05-03T18:47:04.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:04.988+0000 7fd953fff700 1 -- 172.21.15.32:0/1023754064 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fd96400f450 con 0x7fd96c109500 2024-05-03T18:47:04.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:04.989+0000 7fd953fff700 1 --2- 172.21.15.32:0/1023754064 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fd9540b2910 0x7fd9540b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:04.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:04.989+0000 7fd96affd700 1 --2- 172.21.15.32:0/1023754064 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fd9540b2910 0x7fd9540b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:04.990 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:04.990+0000 7fd953fff700 1 -- 172.21.15.32:0/1023754064 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8356+0+0 (secure 0 0 0) 0x7fd9640e23a0 con 0x7fd96c109500 2024-05-03T18:47:04.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:04.990+0000 7fd96affd700 1 --2- 172.21.15.32:0/1023754064 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fd9540b2910 0x7fd9540b4dd0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fd96c19aad0 tx=0x7fd95c00d040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:47:04.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:04.990+0000 7fd97bbafb80 1 Processor -- start 2024-05-03T18:47:04.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:04.990+0000 7fd97bbafb80 1 -- start start 2024-05-03T18:47:04.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:04.990+0000 7fd97bbafb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5639315eac10 0x5639315eaff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:04.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:04.990+0000 7fd97bbafb80 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5639315eb530 0x5639315eecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:04.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:04.990+0000 7fd97bbafb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5639315ef230 0x5639315ef690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:04.992 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:04.990+0000 7fd97bbafb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x56393153d810 con 0x5639315ef230 2024-05-03T18:47:04.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:04.990+0000 7fd97bbafb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5639314dc770 con 0x5639315eac10 2024-05-03T18:47:04.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:04.990+0000 7fd97bbafb80 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5639315397d0 con 0x5639315eb530 2024-05-03T18:47:04.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:04.991+0000 7fd96b7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5639315ef230 0x5639315ef690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:04.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:04.991+0000 7fd96b7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5639315ef230 0x5639315ef690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57450/0 (socket says 172.21.15.32:57450) 2024-05-03T18:47:04.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:04.991+0000 7fd96b7fe700 1 -- 172.21.15.32:0/2339489426 learned_addr learned my addr 172.21.15.32:0/2339489426 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:47:04.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:04.991+0000 7fd96a7fc700 1 --2- 172.21.15.32:0/2339489426 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5639315eb530 0x5639315eecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:04.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:04.991+0000 7fd96affd700 1 --2- 172.21.15.32:0/2339489426 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5639315eac10 0x5639315eaff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:04.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:04.991+0000 7fd96b7fe700 1 -- 172.21.15.32:0/2339489426 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5639315eb530 msgr2=0x5639315eecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:47:04.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:04.991+0000 7fd96b7fe700 1 --2- 172.21.15.32:0/2339489426 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5639315eb530 0x5639315eecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:04.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:04.991+0000 7fd96b7fe700 1 -- 172.21.15.32:0/2339489426 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5639315eac10 msgr2=0x5639315eaff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:47:04.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:04.991+0000 7fd96b7fe700 1 --2- 172.21.15.32:0/2339489426 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5639315eac10 0x5639315eaff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:04.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:04.991+0000 7fd96b7fe700 1 -- 172.21.15.32:0/2339489426 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56393148a880 con 0x5639315ef230 2024-05-03T18:47:04.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:04.991+0000 7fd96affd700 1 --2- 172.21.15.32:0/2339489426 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5639315eac10 0x5639315eaff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:47:04.993 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:04.991+0000 7fd96b7fe700 1 --2- 172.21.15.32:0/2339489426 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5639315ef230 0x5639315ef690 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fd9640a5800 tx=0x7fd9640a9680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f62e7c3af30ac81 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:47:04.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:04.991+0000 7fd9517fa700 1 -- 172.21.15.32:0/2339489426 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9640cc0e0 con 0x5639315ef230 2024-05-03T18:47:04.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:04.991+0000 7fd9517fa700 1 -- 172.21.15.32:0/2339489426 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9640adb30 con 0x5639315ef230 2024-05-03T18:47:04.994 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:04.991+0000 7fd9517fa700 1 -- 172.21.15.32:0/2339489426 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9640b2150 con 0x5639315ef230 2024-05-03T18:47:04.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:04.992+0000 7fd97bbafb80 1 -- 172.21.15.32:0/2339489426 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5639315ef230 msgr2=0x5639315ef690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:04.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:04.992+0000 7fd97bbafb80 1 --2- 172.21.15.32:0/2339489426 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5639315ef230 0x5639315ef690 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fd9640a5800 tx=0x7fd9640a9680 comp rx=0 tx=0).stop 2024-05-03T18:47:04.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:04.992+0000 7fd97bbafb80 1 -- 172.21.15.32:0/2339489426 shutdown_connections 2024-05-03T18:47:04.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:04.992+0000 7fd97bbafb80 1 --2- 172.21.15.32:0/2339489426 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5639315ef230 0x5639315ef690 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:04.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:04.992+0000 7fd97bbafb80 1 --2- 172.21.15.32:0/2339489426 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5639315eac10 0x5639315eaff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:04.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:04.992+0000 7fd97bbafb80 1 --2- 172.21.15.32:0/2339489426 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5639315eb530 0x5639315eecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:04.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:04.992+0000 7fd97bbafb80 1 -- 172.21.15.32:0/2339489426 >> 172.21.15.32:0/2339489426 conn(0x5639315170b0 msgr2=0x5639315e5280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:47:04.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:04.992+0000 7fd97bbafb80 1 -- 172.21.15.32:0/2339489426 shutdown_connections 2024-05-03T18:47:04.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:04.992+0000 7fd97bbafb80 1 -- 172.21.15.32:0/2339489426 wait complete. 2024-05-03T18:47:04.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:04.992+0000 7fd97bbafb80 1 -- start start 2024-05-03T18:47:04.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:04.992+0000 7fd97bbafb80 1 Processor -- start 2024-05-03T18:47:04.998 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:04.993+0000 7fd97bbafb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5639315eac10 0x5639315f6610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:04.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:04.993+0000 7fd97bbafb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5639315eb530 0x5639315f39b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:04.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:04.993+0000 7fd97bbafb80 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5639315ef230 0x5639315f3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:04.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:04.993+0000 7fd97bbafb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5639315f7490 con 0x5639315eac10 2024-05-03T18:47:04.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:04.993+0000 7fd97bbafb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5639314a3be0 con 0x5639315eb530 2024-05-03T18:47:04.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:04.993+0000 7fd97bbafb80 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5639315f7610 con 0x5639315ef230 2024-05-03T18:47:04.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:04.993+0000 7fd97bbafb80 10 client.0 _ll_register_callbacks cb 0x5639314d58d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:47:04.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:04.993+0000 7fd96affd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5639315eac10 0x5639315f6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:04.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:04.993+0000 7fd96affd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5639315eac10 0x5639315f6610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57466/0 (socket says 172.21.15.32:57466) 2024-05-03T18:47:04.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:04.993+0000 7fd96affd700 1 -- 172.21.15.32:0/4232628890 learned_addr learned my addr 172.21.15.32:0/4232628890 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:47:04.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:04.993+0000 7fd96a7fc700 1 --2- 172.21.15.32:0/4232628890 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5639315eb530 0x5639315f39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:04.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:04.993+0000 7fd96b7fe700 1 --2- 172.21.15.32:0/4232628890 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5639315ef230 0x5639315f3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:04.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:04.993+0000 7fd96affd700 1 -- 172.21.15.32:0/4232628890 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5639315ef230 msgr2=0x5639315f3ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:47:04.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:04.993+0000 7fd96affd700 1 --2- 172.21.15.32:0/4232628890 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5639315ef230 0x5639315f3ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:05.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:04.993+0000 7fd96affd700 1 -- 172.21.15.32:0/4232628890 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5639315eb530 msgr2=0x5639315f39b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:47:05.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:04.993+0000 7fd96affd700 1 --2- 172.21.15.32:0/4232628890 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5639315eb530 0x5639315f39b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:05.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:04.993+0000 7fd96affd700 1 -- 172.21.15.32:0/4232628890 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5639314d9640 con 0x5639315eac10 2024-05-03T18:47:05.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:04.993+0000 7fd96affd700 1 --2- 172.21.15.32:0/4232628890 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5639315eac10 0x5639315f6610 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fd95c016e10 tx=0x7fd95c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f42b1acd233072d9 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:47:05.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:04.993+0000 7fd94eff5700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-05-03T18:47:05.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:04.993+0000 7fd94eff5700 1 -- 172.21.15.32:0/4232628890 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd95c00ae40 con 0x5639315eac10 2024-05-03T18:47:05.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:04.993+0000 7fd94eff5700 1 -- 172.21.15.32:0/4232628890 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8356+0+0 (secure 0 0 0) 0x7fd95c021460 con 0x5639315eac10 2024-05-03T18:47:05.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:04.993+0000 7fd97bbafb80 1 -- 172.21.15.32:0/4232628890 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd9640adb30 con 0x5639315eac10 2024-05-03T18:47:05.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:04.994+0000 7fd94eff5700 1 -- 172.21.15.32:0/4232628890 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd95c021660 con 0x5639315eac10 2024-05-03T18:47:05.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:04.994+0000 7fd97bbafb80 10 client.15012 fetch_fsmap learned FSMap version 25 2024-05-03T18:47:05.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:04.994+0000 7fd97bbafb80 1 -- 172.21.15.32:0/4232628890 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x563931601bd0 con 0x5639315eac10 2024-05-03T18:47:05.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:04.994+0000 7fd94eff5700 1 -- 172.21.15.32:0/4232628890 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9640adb30 con 0x5639315eac10 2024-05-03T18:47:05.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:04.994+0000 7fd94eff5700 1 -- 172.21.15.32:0/4232628890 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fd95c02a960 con 0x5639315eac10 2024-05-03T18:47:05.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:04.994+0000 7fd97bbafb80 10 client.15012 fetch_fsmap finished waiting for FSMap version 25 2024-05-03T18:47:05.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:04.994+0000 7fd97bbafb80 10 client.15012 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-03T18:47:05.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:04.994+0000 7fd97bbafb80 20 client.15012 populate_metadata read hostname 'smithi032' 2024-05-03T18:47:05.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:04.994+0000 7fd97bbafb80 1 --2- 172.21.15.32:0/4232628890 >> [v2:172.21.15.113:6838/942050693,v1:172.21.15.113:6839/942050693] conn(0x563931604250 0x5639316056c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:05.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:04.994+0000 7fd97bbafb80 4 client.15012 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-03T18:47:05.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:04.994+0000 7fd97bbafb80 1 -- 172.21.15.32:0/4232628890 --> [v2:172.21.15.113:6838/942050693,v1:172.21.15.113:6839/942050693] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56393154b1f0 con 0x563931604250 2024-05-03T18:47:05.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:04.994+0000 7fd96a7fc700 1 --2- 172.21.15.32:0/4232628890 >> [v2:172.21.15.113:6838/942050693,v1:172.21.15.113:6839/942050693] conn(0x563931604250 0x5639316056c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:05.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:04.995+0000 7fd96a7fc700 1 --2- 172.21.15.32:0/4232628890 >> [v2:172.21.15.113:6838/942050693,v1:172.21.15.113:6839/942050693] conn(0x563931604250 0x5639316056c0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d6d8ed1d1c18103 server_cookie=f96c420dff1c1407 in_seq=0 out_seq=0 2024-05-03T18:47:05.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:04.995+0000 7fd94eff5700 10 client.15012 ms_handle_connect on v2:172.21.15.113:6838/942050693 2024-05-03T18:47:05.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:04.996+0000 7fd94eff5700 1 -- 172.21.15.32:0/4232628890 <== mds.0 v2:172.21.15.113:6838/942050693 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56393154b1f0 con 0x563931604250 2024-05-03T18:47:05.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:04.996+0000 7fd94eff5700 10 client.15012 handle_command_reply: tid=0 2024-05-03T18:47:05.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:04.996+0000 7fd97bbafb80 1 client.15012 shutdown 2024-05-03T18:47:05.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:04.996+0000 7fd97bbafb80 1 -- 172.21.15.32:0/4232628890 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5639315eac10 msgr2=0x5639315f6610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:05.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:04.996+0000 7fd97bbafb80 1 --2- 172.21.15.32:0/4232628890 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5639315eac10 0x5639315f6610 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fd95c016e10 tx=0x7fd95c004ad0 comp rx=0 tx=0).stop 2024-05-03T18:47:05.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:04.996+0000 7fd97bbafb80 1 -- 172.21.15.32:0/4232628890 shutdown_connections 2024-05-03T18:47:05.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:04.996+0000 7fd97bbafb80 1 --2- 172.21.15.32:0/4232628890 >> [v2:172.21.15.113:6838/942050693,v1:172.21.15.113:6839/942050693] conn(0x563931604250 0x5639316056c0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:05.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:04.996+0000 7fd94eff5700 0 client.15012 ms_handle_reset on v2:172.21.15.113:6838/942050693 2024-05-03T18:47:05.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:04.996+0000 7fd97bbafb80 1 --2- 172.21.15.32:0/4232628890 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5639315ef230 0x5639315f3ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:05.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:04.996+0000 7fd97bbafb80 1 --2- 172.21.15.32:0/4232628890 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5639315eac10 0x5639315f6610 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:05.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:04.996+0000 7fd97bbafb80 1 --2- 172.21.15.32:0/4232628890 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5639315eb530 0x5639315f39b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:05.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:04.996+0000 7fd97bbafb80 1 -- 172.21.15.32:0/4232628890 >> 172.21.15.32:0/4232628890 conn(0x5639315170b0 msgr2=0x5639315f06e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:47:05.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:04.996+0000 7fd97bbafb80 1 -- 172.21.15.32:0/4232628890 shutdown_connections 2024-05-03T18:47:05.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:04.996+0000 7fd97bbafb80 1 -- 172.21.15.32:0/4232628890 wait complete. 2024-05-03T18:47:05.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:04.996+0000 7fd97bbafb80 20 client.15012 trim_cache size 0 max 16384 2024-05-03T18:47:05.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:05.017+0000 7fd97bbafb80 1 Processor -- start 2024-05-03T18:47:05.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:05.017+0000 7fd97bbafb80 1 -- start start 2024-05-03T18:47:05.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:05.017+0000 7fd97bbafb80 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5639315170b0 0x5639315f2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:05.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:05.017+0000 7fd97bbafb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5639315eac10 0x56393160ef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:05.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:05.017+0000 7fd97bbafb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5639315eb530 0x5639315f3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:05.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:05.018+0000 7fd97bbafb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x563931603a00 con 0x5639315eac10 2024-05-03T18:47:05.019 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:05.018+0000 7fd97bbafb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5639316040d0 con 0x5639315eb530 2024-05-03T18:47:05.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:05.018+0000 7fd97bbafb80 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5639316047a0 con 0x5639315170b0 2024-05-03T18:47:05.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:05.018+0000 7fd96a7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5639315eac10 0x56393160ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:05.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:05.018+0000 7fd96a7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5639315eac10 0x56393160ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57482/0 (socket says 172.21.15.32:57482) 2024-05-03T18:47:05.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:05.018+0000 7fd96a7fc700 1 -- 172.21.15.32:0/3703824588 learned_addr learned my addr 172.21.15.32:0/3703824588 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:47:05.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:05.018+0000 7fd96affd700 1 --2- 172.21.15.32:0/3703824588 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5639315170b0 0x5639315f2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:05.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:05.018+0000 7fd96a7fc700 1 -- 172.21.15.32:0/3703824588 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5639315170b0 msgr2=0x5639315f2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:47:05.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:05.018+0000 7fd96a7fc700 1 --2- 172.21.15.32:0/3703824588 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5639315170b0 0x5639315f2930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:05.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:05.018+0000 7fd96a7fc700 1 -- 172.21.15.32:0/3703824588 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5639315eb530 msgr2=0x5639315f3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:47:05.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:05.018+0000 7fd96a7fc700 1 --2- 172.21.15.32:0/3703824588 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5639315eb530 0x5639315f3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:05.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:05.018+0000 7fd96a7fc700 1 -- 172.21.15.32:0/3703824588 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639315e01f0 con 0x5639315eac10 2024-05-03T18:47:05.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:05.018+0000 7fd96a7fc700 1 --2- 172.21.15.32:0/3703824588 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5639315eac10 0x56393160ef10 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fd960000c00 tx=0x7fd96001a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39c843722e6a241 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:47:05.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:05.018+0000 7fd950ff9700 1 -- 172.21.15.32:0/3703824588 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9600050b0 con 0x5639315eac10 2024-05-03T18:47:05.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:05.018+0000 7fd950ff9700 1 -- 172.21.15.32:0/3703824588 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd96000b770 con 0x5639315eac10 2024-05-03T18:47:05.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:05.018+0000 7fd950ff9700 1 -- 172.21.15.32:0/3703824588 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9600293b0 con 0x5639315eac10 2024-05-03T18:47:05.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:05.019+0000 7fd97bbafb80 1 -- 172.21.15.32:0/3703824588 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5639315eac10 msgr2=0x56393160ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:05.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:05.019+0000 7fd97bbafb80 1 --2- 172.21.15.32:0/3703824588 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5639315eac10 0x56393160ef10 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fd960000c00 tx=0x7fd96001a040 comp rx=0 tx=0).stop 2024-05-03T18:47:05.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:05.019+0000 7fd97bbafb80 1 -- 172.21.15.32:0/3703824588 shutdown_connections 2024-05-03T18:47:05.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:05.019+0000 7fd97bbafb80 1 --2- 172.21.15.32:0/3703824588 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5639315eb530 0x5639315f3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:05.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:05.019+0000 7fd97bbafb80 1 --2- 172.21.15.32:0/3703824588 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5639315170b0 0x5639315f2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:05.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:05.019+0000 7fd97bbafb80 1 --2- 172.21.15.32:0/3703824588 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5639315eac10 0x56393160ef10 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:05.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:05.019+0000 7fd97bbafb80 1 -- 172.21.15.32:0/3703824588 >> 172.21.15.32:0/3703824588 conn(0x5639315e1eb0 msgr2=0x5639315ea2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:47:05.028 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:05.019+0000 7fd97bbafb80 1 -- 172.21.15.32:0/3703824588 shutdown_connections 2024-05-03T18:47:05.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:05.019+0000 7fd97bbafb80 1 -- 172.21.15.32:0/3703824588 wait complete. 2024-05-03T18:47:05.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:05.019+0000 7fd97bbafb80 1 -- start start 2024-05-03T18:47:05.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:05.019+0000 7fd97bbafb80 1 Processor -- start 2024-05-03T18:47:05.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:05.019+0000 7fd97bbafb80 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5639315170b0 0x56393160bef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:05.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:05.019+0000 7fd97bbafb80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5639315eac10 0x56393160c430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:05.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:05.019+0000 7fd97bbafb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5639315eb530 0x56393160c970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:05.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:05.020+0000 7fd97bbafb80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x563931605540 con 0x5639315eac10 2024-05-03T18:47:05.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:05.020+0000 7fd97bbafb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x563931604e70 con 0x5639315eb530 2024-05-03T18:47:05.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:05.020+0000 7fd97bbafb80 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x563931605c10 con 0x5639315170b0 2024-05-03T18:47:05.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:05.020+0000 7fd97bbafb80 10 client.0 _ll_register_callbacks cb 0x5639314d58d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:47:05.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:05.020+0000 7fd96a7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5639315eac10 0x56393160c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:05.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:05.020+0000 7fd96a7fc700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5639315eac10 0x56393160c430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57492/0 (socket says 172.21.15.32:57492) 2024-05-03T18:47:05.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:05.020+0000 7fd96a7fc700 1 -- 172.21.15.32:0/1794322163 learned_addr learned my addr 172.21.15.32:0/1794322163 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:47:05.029 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:05.020+0000 7fd96affd700 1 --2- 172.21.15.32:0/1794322163 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5639315170b0 0x56393160bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:05.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:05.020+0000 7fd96a7fc700 1 -- 172.21.15.32:0/1794322163 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5639315170b0 msgr2=0x56393160bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:47:05.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:05.020+0000 7fd96a7fc700 1 --2- 172.21.15.32:0/1794322163 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5639315170b0 0x56393160bef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:05.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:05.020+0000 7fd96b7fe700 1 --2- 172.21.15.32:0/1794322163 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5639315eb530 0x56393160c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:05.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:05.020+0000 7fd96a7fc700 1 -- 172.21.15.32:0/1794322163 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5639315eb530 msgr2=0x56393160c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:47:05.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:05.020+0000 7fd96a7fc700 1 --2- 172.21.15.32:0/1794322163 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5639315eb530 0x56393160c970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:05.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:05.020+0000 7fd96a7fc700 1 -- 172.21.15.32:0/1794322163 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563931673860 con 0x5639315eac10 2024-05-03T18:47:05.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:05.020+0000 7fd96affd700 1 --2- 172.21.15.32:0/1794322163 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5639315170b0 0x56393160bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:47:05.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:05.020+0000 7fd94eff5700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-05-03T18:47:05.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:05.020+0000 7fd96a7fc700 1 --2- 172.21.15.32:0/1794322163 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5639315eac10 0x56393160c430 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fd960003b90 tx=0x7fd96000b9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f5fcca6376b92e server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:47:05.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:05.020+0000 7fd94eff5700 1 -- 172.21.15.32:0/1794322163 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9600052b0 con 0x5639315eac10 2024-05-03T18:47:05.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:05.021+0000 7fd94eff5700 1 -- 172.21.15.32:0/1794322163 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8356+0+0 (secure 0 0 0) 0x7fd960005450 con 0x5639315eac10 2024-05-03T18:47:05.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:05.021+0000 7fd97bbafb80 1 -- 172.21.15.32:0/1794322163 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd96000b770 con 0x5639315eac10 2024-05-03T18:47:05.030 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:05.021+0000 7fd94eff5700 1 -- 172.21.15.32:0/1794322163 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd960043050 con 0x5639315eac10 2024-05-03T18:47:05.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:05.021+0000 7fd97bbafb80 10 client.15018 fetch_fsmap learned FSMap version 25 2024-05-03T18:47:05.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:05.021+0000 7fd97bbafb80 1 -- 172.21.15.32:0/1794322163 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x563931674b20 con 0x5639315eac10 2024-05-03T18:47:05.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:05.021+0000 7fd94eff5700 1 -- 172.21.15.32:0/1794322163 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd96000b770 con 0x5639315eac10 2024-05-03T18:47:05.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:05.021+0000 7fd94eff5700 1 -- 172.21.15.32:0/1794322163 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fd96000f4a0 con 0x5639315eac10 2024-05-03T18:47:05.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:05.021+0000 7fd97bbafb80 10 client.15018 fetch_fsmap finished waiting for FSMap version 25 2024-05-03T18:47:05.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:05.021+0000 7fd97bbafb80 10 client.15018 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-03T18:47:05.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:05.021+0000 7fd97bbafb80 20 client.15018 populate_metadata read hostname 'smithi032' 2024-05-03T18:47:05.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:05.021+0000 7fd97bbafb80 1 --2- 172.21.15.32:0/1794322163 >> [v2:172.21.15.113:6838/942050693,v1:172.21.15.113:6839/942050693] conn(0x5639315ef230 0x563931677e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:05.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:05.022+0000 7fd97bbafb80 4 client.15018 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-03T18:47:05.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:05.022+0000 7fd97bbafb80 1 -- 172.21.15.32:0/1794322163 --> [v2:172.21.15.113:6838/942050693,v1:172.21.15.113:6839/942050693] -- command(tid 0: {"prefix": "status"}) -- 0x563931545290 con 0x5639315ef230 2024-05-03T18:47:05.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:05.022+0000 7fd96affd700 1 --2- 172.21.15.32:0/1794322163 >> [v2:172.21.15.113:6838/942050693,v1:172.21.15.113:6839/942050693] conn(0x5639315ef230 0x563931677e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:05.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:05.022+0000 7fd96affd700 1 --2- 172.21.15.32:0/1794322163 >> [v2:172.21.15.113:6838/942050693,v1:172.21.15.113:6839/942050693] conn(0x5639315ef230 0x563931677e70 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=43875de3ca2dda20 server_cookie=f190b7a6e1d35532 in_seq=0 out_seq=0 2024-05-03T18:47:05.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:05.022+0000 7fd94eff5700 10 client.15018 ms_handle_connect on v2:172.21.15.113:6838/942050693 2024-05-03T18:47:05.031 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:05.023+0000 7fd94eff5700 1 -- 172.21.15.32:0/1794322163 <== mds.0 v2:172.21.15.113:6838/942050693 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x563931545290 con 0x5639315ef230 2024-05-03T18:47:05.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:05.023+0000 7fd94eff5700 10 client.15018 handle_command_reply: tid=0 2024-05-03T18:47:05.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:05.023+0000 7fd97bbafb80 1 client.15018 shutdown 2024-05-03T18:47:05.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:05.023+0000 7fd97bbafb80 1 -- 172.21.15.32:0/1794322163 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5639315eac10 msgr2=0x56393160c430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:05.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:05.023+0000 7fd97bbafb80 1 --2- 172.21.15.32:0/1794322163 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5639315eac10 0x56393160c430 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fd960003b90 tx=0x7fd96000b9d0 comp rx=0 tx=0).stop 2024-05-03T18:47:05.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:05.023+0000 7fd97bbafb80 1 -- 172.21.15.32:0/1794322163 shutdown_connections 2024-05-03T18:47:05.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:05.023+0000 7fd97bbafb80 1 --2- 172.21.15.32:0/1794322163 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5639315eb530 0x56393160c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:05.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:05.023+0000 7fd97bbafb80 1 --2- 172.21.15.32:0/1794322163 >> [v2:172.21.15.113:6838/942050693,v1:172.21.15.113:6839/942050693] conn(0x5639315ef230 0x563931677e70 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:05.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:05.023+0000 7fd97bbafb80 1 --2- 172.21.15.32:0/1794322163 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5639315170b0 0x56393160bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:05.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:05.023+0000 7fd94eff5700 0 client.15018 ms_handle_reset on v2:172.21.15.113:6838/942050693 2024-05-03T18:47:05.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:05.023+0000 7fd97bbafb80 1 --2- 172.21.15.32:0/1794322163 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5639315eac10 0x56393160c430 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:05.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:05.023+0000 7fd97bbafb80 1 -- 172.21.15.32:0/1794322163 >> 172.21.15.32:0/1794322163 conn(0x5639315e1eb0 msgr2=0x5639315ff8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:47:05.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:05.023+0000 7fd97bbafb80 1 -- 172.21.15.32:0/1794322163 shutdown_connections 2024-05-03T18:47:05.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:05.023+0000 7fd97bbafb80 1 -- 172.21.15.32:0/1794322163 wait complete. 2024-05-03T18:47:05.032 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:05.023+0000 7fd97bbafb80 20 client.15018 trim_cache size 0 max 16384 2024-05-03T18:47:05.033 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-05-03T18:47:05.033 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_fsid": "f500455e-097b-11ef-bc97-c7b262605968", 2024-05-03T18:47:05.033 INFO:teuthology.orchestra.run.smithi032.stdout: "whoami": 0, 2024-05-03T18:47:05.033 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 24478, 2024-05-03T18:47:05.033 INFO:teuthology.orchestra.run.smithi032.stdout: "want_state": "up:active", 2024-05-03T18:47:05.033 INFO:teuthology.orchestra.run.smithi032.stdout: "state": "up:active", 2024-05-03T18:47:05.033 INFO:teuthology.orchestra.run.smithi032.stdout: "fs_name": "cephfs", 2024-05-03T18:47:05.033 INFO:teuthology.orchestra.run.smithi032.stdout: "rank_uptime": 133.60261351, 2024-05-03T18:47:05.033 INFO:teuthology.orchestra.run.smithi032.stdout: "mdsmap_epoch": 25, 2024-05-03T18:47:05.033 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch": 82, 2024-05-03T18:47:05.034 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch_barrier": 78, 2024-05-03T18:47:05.034 INFO:teuthology.orchestra.run.smithi032.stdout: "uptime": 157.29206571099999 2024-05-03T18:47:05.034 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-05-03T18:47:05.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:05.024+0000 7fd97131c700 1 -- 172.21.15.32:0/1023754064 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fd9540b2910 msgr2=0x7fd9540b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:05.034 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:05.024+0000 7fd97131c700 1 --2- 172.21.15.32:0/1023754064 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fd9540b2910 0x7fd9540b4dd0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fd96c19aad0 tx=0x7fd95c00d040 comp rx=0 tx=0).stop 2024-05-03T18:47:05.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:05.024+0000 7fd97131c700 1 -- 172.21.15.32:0/1023754064 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd96c109500 msgr2=0x7fd96c199990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:05.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:05.024+0000 7fd97131c700 1 --2- 172.21.15.32:0/1023754064 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd96c109500 0x7fd96c199990 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fd96400c900 tx=0x7fd96400cc10 comp rx=0 tx=0).stop 2024-05-03T18:47:05.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:05.024+0000 7fd97131c700 1 -- 172.21.15.32:0/1023754064 shutdown_connections 2024-05-03T18:47:05.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:05.024+0000 7fd97131c700 1 --2- 172.21.15.32:0/1023754064 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fd9540b2910 0x7fd9540b4dd0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:05.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:05.024+0000 7fd97131c700 1 --2- 172.21.15.32:0/1023754064 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd96c109500 0x7fd96c199990 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:05.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:05.024+0000 7fd97131c700 1 --2- 172.21.15.32:0/1023754064 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd96c06ba10 0x7fd96c19f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:05.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:05.024+0000 7fd97131c700 1 --2- 172.21.15.32:0/1023754064 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd96c06c3d0 0x7fd96c19fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:05.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:05.024+0000 7fd97131c700 1 -- 172.21.15.32:0/1023754064 >> 172.21.15.32:0/1023754064 conn(0x7fd96c079310 msgr2=0x7fd96c101020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:47:05.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:05.024+0000 7fd97131c700 1 -- 172.21.15.32:0/1023754064 shutdown_connections 2024-05-03T18:47:05.035 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:05.024+0000 7fd97131c700 1 -- 172.21.15.32:0/1023754064 wait complete. 2024-05-03T18:47:05.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:47:05 smithi032 ceph-mon[29763]: pgmap v351: 97 pgs: 97 active+clean; 631 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:47:05.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:47:05 smithi113 ceph-mon[33540]: pgmap v351: 97 pgs: 97 active+clean; 631 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:47:05.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:47:05 smithi045 ceph-mon[33299]: pgmap v351: 97 pgs: 97 active+clean; 631 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:47:05.791 INFO:tasks.cephadm:Setup complete, yielding 2024-05-03T18:47:05.791 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-03T18:47:05.801 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi032.front.sepia.ceph.com 2024-05-03T18:47:05.801 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f500455e-097b-11ef-bc97-c7b262605968 -- bash -c 'ceph orch status' 2024-05-03T18:47:06.051 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:47:07.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:47:07 smithi113 ceph-mon[33540]: pgmap v352: 97 pgs: 97 active+clean; 631 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:47:07.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:47:07 smithi032 ceph-mon[29763]: pgmap v352: 97 pgs: 97 active+clean; 631 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:47:07.716 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:07.713+0000 7f3012824700 1 -- 172.21.15.32:0/2602205511 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f300c106920 msgr2=0x7f300c108d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:07.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:07.713+0000 7f3012824700 1 --2- 172.21.15.32:0/2602205511 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f300c106920 0x7f300c108d10 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f2ff4009990 tx=0x7f2ff4009ca0 comp rx=0 tx=0).stop 2024-05-03T18:47:07.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:07.713+0000 7f3012824700 1 -- 172.21.15.32:0/2602205511 shutdown_connections 2024-05-03T18:47:07.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:07.713+0000 7f3012824700 1 --2- 172.21.15.32:0/2602205511 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f300c109250 0x7f300c10b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:07.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:07.713+0000 7f3012824700 1 --2- 172.21.15.32:0/2602205511 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f300c103ff0 0x7f300c1063e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:07.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:07.713+0000 7f3012824700 1 --2- 172.21.15.32:0/2602205511 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f300c106920 0x7f300c108d10 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:07.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:07.713+0000 7f3012824700 1 -- 172.21.15.32:0/2602205511 >> 172.21.15.32:0/2602205511 conn(0x7f300c0fd9a0 msgr2=0x7f300c0ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:47:07.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:07.713+0000 7f3012824700 1 -- 172.21.15.32:0/2602205511 shutdown_connections 2024-05-03T18:47:07.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:07.713+0000 7f3012824700 1 -- 172.21.15.32:0/2602205511 wait complete. 2024-05-03T18:47:07.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:07.714+0000 7f3012824700 1 Processor -- start 2024-05-03T18:47:07.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:07.714+0000 7f3012824700 1 -- start start 2024-05-03T18:47:07.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:07.714+0000 7f3012824700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f300c103ff0 0x7f300c19b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:07.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:07.714+0000 7f3012824700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f300c106920 0x7f300c19be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:07.717 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:07.714+0000 7f3012824700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f300c109250 0x7f300c1a2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:07.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:07.714+0000 7f3012824700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f300c10f000 con 0x7f300c109250 2024-05-03T18:47:07.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:07.714+0000 7f3012824700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f300c10ee80 con 0x7f300c106920 2024-05-03T18:47:07.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:07.714+0000 7f3012824700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f300c10f180 con 0x7f300c103ff0 2024-05-03T18:47:07.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:07.714+0000 7f3010dc1700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f300c109250 0x7f300c1a2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:07.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:07.714+0000 7f3010dc1700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f300c109250 0x7f300c1a2eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57508/0 (socket says 172.21.15.32:57508) 2024-05-03T18:47:07.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:07.714+0000 7f3010dc1700 1 -- 172.21.15.32:0/98001363 learned_addr learned my addr 172.21.15.32:0/98001363 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:47:07.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:07.714+0000 7f300b7fe700 1 --2- 172.21.15.32:0/98001363 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f300c106920 0x7f300c19be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:07.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:07.714+0000 7f300bfff700 1 --2- 172.21.15.32:0/98001363 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f300c103ff0 0x7f300c19b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:07.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:07.714+0000 7f3010dc1700 1 -- 172.21.15.32:0/98001363 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f300c103ff0 msgr2=0x7f300c19b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:07.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:07.714+0000 7f3010dc1700 1 --2- 172.21.15.32:0/98001363 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f300c103ff0 0x7f300c19b8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:07.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:07.714+0000 7f3010dc1700 1 -- 172.21.15.32:0/98001363 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f300c106920 msgr2=0x7f300c19be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:07.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:07.714+0000 7f3010dc1700 1 --2- 172.21.15.32:0/98001363 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f300c106920 0x7f300c19be30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:07.718 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:07.714+0000 7f3010dc1700 1 -- 172.21.15.32:0/98001363 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f300c1a3550 con 0x7f300c109250 2024-05-03T18:47:07.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:07.715+0000 7f3010dc1700 1 --2- 172.21.15.32:0/98001363 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f300c109250 0x7f300c1a2eb0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f300400ba10 tx=0x7f300400bdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:47:07.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:07.715+0000 7f30097fa700 1 -- 172.21.15.32:0/98001363 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3004019070 con 0x7f300c109250 2024-05-03T18:47:07.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:07.715+0000 7f3012824700 1 -- 172.21.15.32:0/98001363 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f300c1a3840 con 0x7f300c109250 2024-05-03T18:47:07.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:07.715+0000 7f30097fa700 1 -- 172.21.15.32:0/98001363 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3004014440 con 0x7f300c109250 2024-05-03T18:47:07.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:07.715+0000 7f30097fa700 1 -- 172.21.15.32:0/98001363 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f300400ce60 con 0x7f300c109250 2024-05-03T18:47:07.719 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:07.715+0000 7f3012824700 1 -- 172.21.15.32:0/98001363 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f300c1a3e50 con 0x7f300c109250 2024-05-03T18:47:07.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:07.716+0000 7f3012824700 1 -- 172.21.15.32:0/98001363 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f300c06a190 con 0x7f300c109250 2024-05-03T18:47:07.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:07.717+0000 7f30097fa700 1 -- 172.21.15.32:0/98001363 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f300401c070 con 0x7f300c109250 2024-05-03T18:47:07.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:07.718+0000 7f30097fa700 1 --2- 172.21.15.32:0/98001363 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f2ff80b2910 0x7f2ff80b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:07.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:07.718+0000 7f300bfff700 1 --2- 172.21.15.32:0/98001363 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f2ff80b2910 0x7f2ff80b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:07.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:07.718+0000 7f30097fa700 1 -- 172.21.15.32:0/98001363 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8356+0+0 (secure 0 0 0) 0x7f30040e1b10 con 0x7f300c109250 2024-05-03T18:47:07.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:07.718+0000 7f300bfff700 1 --2- 172.21.15.32:0/98001363 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f2ff80b2910 0x7f2ff80b4dd0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc005950 tx=0x7f2ffc0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:47:07.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:07.722+0000 7f30097fa700 1 -- 172.21.15.32:0/98001363 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f30040a97f0 con 0x7f300c109250 2024-05-03T18:47:07.747 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:47:07 smithi045 ceph-mon[33299]: pgmap v352: 97 pgs: 97 active+clean; 631 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:47:07.953 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:07.952+0000 7f3012824700 1 -- 172.21.15.32:0/98001363 --> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f300c0638e0 con 0x7f2ff80b2910 2024-05-03T18:47:07.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:07.954+0000 7f30097fa700 1 -- 172.21.15.32:0/98001363 <== mgr.14152 v2:172.21.15.32:6800/1796547904 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f300c0638e0 con 0x7f2ff80b2910 2024-05-03T18:47:07.955 INFO:teuthology.orchestra.run.smithi032.stdout:Backend: cephadm 2024-05-03T18:47:07.955 INFO:teuthology.orchestra.run.smithi032.stdout:Available: Yes 2024-05-03T18:47:07.955 INFO:teuthology.orchestra.run.smithi032.stdout:Paused: No 2024-05-03T18:47:07.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:07.956+0000 7f3012824700 1 -- 172.21.15.32:0/98001363 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f2ff80b2910 msgr2=0x7f2ff80b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:07.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:07.956+0000 7f3012824700 1 --2- 172.21.15.32:0/98001363 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f2ff80b2910 0x7f2ff80b4dd0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc005950 tx=0x7f2ffc0058e0 comp rx=0 tx=0).stop 2024-05-03T18:47:07.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:07.956+0000 7f3012824700 1 -- 172.21.15.32:0/98001363 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f300c109250 msgr2=0x7f300c1a2eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:07.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:07.956+0000 7f3012824700 1 --2- 172.21.15.32:0/98001363 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f300c109250 0x7f300c1a2eb0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f300400ba10 tx=0x7f300400bdd0 comp rx=0 tx=0).stop 2024-05-03T18:47:07.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:07.956+0000 7f3012824700 1 -- 172.21.15.32:0/98001363 shutdown_connections 2024-05-03T18:47:07.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:07.956+0000 7f3012824700 1 --2- 172.21.15.32:0/98001363 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f2ff80b2910 0x7f2ff80b4dd0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:07.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:07.956+0000 7f3012824700 1 --2- 172.21.15.32:0/98001363 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f300c109250 0x7f300c1a2eb0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:07.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:07.956+0000 7f3012824700 1 --2- 172.21.15.32:0/98001363 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f300c103ff0 0x7f300c19b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:07.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:07.956+0000 7f3012824700 1 --2- 172.21.15.32:0/98001363 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f300c106920 0x7f300c19be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:07.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:07.956+0000 7f3012824700 1 -- 172.21.15.32:0/98001363 >> 172.21.15.32:0/98001363 conn(0x7f300c0fd9a0 msgr2=0x7f300c1036e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:47:07.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:07.956+0000 7f3012824700 1 -- 172.21.15.32:0/98001363 shutdown_connections 2024-05-03T18:47:07.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:07.956+0000 7f3012824700 1 -- 172.21.15.32:0/98001363 wait complete. 2024-05-03T18:47:08.616 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f500455e-097b-11ef-bc97-c7b262605968 -- bash -c 'ceph orch ps' 2024-05-03T18:47:08.862 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:47:09.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:47:09 smithi113 ceph-mon[33540]: from='client.15024 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:47:09.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:47:09 smithi113 ceph-mon[33540]: pgmap v353: 97 pgs: 97 active+clean; 631 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:47:09.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:47:09 smithi032 ceph-mon[29763]: from='client.15024 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:47:09.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:47:09 smithi032 ceph-mon[29763]: pgmap v353: 97 pgs: 97 active+clean; 631 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:47:09.747 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:47:09 smithi045 ceph-mon[33299]: from='client.15024 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:47:09.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:47:09 smithi045 ceph-mon[33299]: pgmap v353: 97 pgs: 97 active+clean; 631 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:47:10.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:10.448+0000 7f5219b75700 1 -- 172.21.15.32:0/3568162715 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5214103600 msgr2=0x7f5214103a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:10.456 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:10.448+0000 7f5219b75700 1 --2- 172.21.15.32:0/3568162715 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5214103600 0x7f5214103a60 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f52080099c0 tx=0x7f5208009cd0 comp rx=0 tx=0).stop 2024-05-03T18:47:10.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:10.448+0000 7f5219b75700 1 -- 172.21.15.32:0/3568162715 shutdown_connections 2024-05-03T18:47:10.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:10.448+0000 7f5219b75700 1 --2- 172.21.15.32:0/3568162715 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5214103fa0 0x7f521410a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:10.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:10.448+0000 7f5219b75700 1 --2- 172.21.15.32:0/3568162715 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5214102400 0x7f5214102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:10.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:10.448+0000 7f5219b75700 1 --2- 172.21.15.32:0/3568162715 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5214103600 0x7f5214103a60 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:10.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:10.448+0000 7f5219b75700 1 -- 172.21.15.32:0/3568162715 >> 172.21.15.32:0/3568162715 conn(0x7f52140fd9a0 msgr2=0x7f52140ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:47:10.457 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:10.448+0000 7f5219b75700 1 -- 172.21.15.32:0/3568162715 shutdown_connections 2024-05-03T18:47:10.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:10.448+0000 7f5219b75700 1 -- 172.21.15.32:0/3568162715 wait complete. 2024-05-03T18:47:10.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:10.449+0000 7f5219b75700 1 Processor -- start 2024-05-03T18:47:10.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:10.449+0000 7f5219b75700 1 -- start start 2024-05-03T18:47:10.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:10.449+0000 7f5219b75700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5214102400 0x7f52141984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:10.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:10.449+0000 7f5219b75700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5214103600 0x7f5214198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:10.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:10.449+0000 7f5219b75700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5214103fa0 0x7f521419fab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:10.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:10.449+0000 7f5219b75700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f521410de80 con 0x7f5214102400 2024-05-03T18:47:10.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:10.449+0000 7f5219b75700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f521410dd00 con 0x7f5214103600 2024-05-03T18:47:10.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:10.449+0000 7f5219b75700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f521410e000 con 0x7f5214103fa0 2024-05-03T18:47:10.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:10.449+0000 7f52137fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5214102400 0x7f52141984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:10.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:10.449+0000 7f52137fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5214102400 0x7f52141984f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57530/0 (socket says 172.21.15.32:57530) 2024-05-03T18:47:10.459 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:10.449+0000 7f52137fe700 1 -- 172.21.15.32:0/4057580654 learned_addr learned my addr 172.21.15.32:0/4057580654 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:47:10.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:10.450+0000 7f5212ffd700 1 --2- 172.21.15.32:0/4057580654 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5214103600 0x7f5214198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:10.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:10.450+0000 7f5213fff700 1 --2- 172.21.15.32:0/4057580654 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5214103fa0 0x7f521419fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:10.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:10.450+0000 7f52137fe700 1 -- 172.21.15.32:0/4057580654 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5214103fa0 msgr2=0x7f521419fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:10.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:10.450+0000 7f52137fe700 1 --2- 172.21.15.32:0/4057580654 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5214103fa0 0x7f521419fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:10.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:10.450+0000 7f52137fe700 1 -- 172.21.15.32:0/4057580654 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5214103600 msgr2=0x7f5214198a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:10.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:10.450+0000 7f52137fe700 1 --2- 172.21.15.32:0/4057580654 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5214103600 0x7f5214198a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:10.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:10.450+0000 7f52137fe700 1 -- 172.21.15.32:0/4057580654 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52141a0150 con 0x7f5214102400 2024-05-03T18:47:10.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:10.450+0000 7f52137fe700 1 --2- 172.21.15.32:0/4057580654 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5214102400 0x7f52141984f0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f520400c9c0 tx=0x7f520400cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:47:10.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:10.450+0000 7f5210ff9700 1 -- 172.21.15.32:0/4057580654 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5204007ac0 con 0x7f5214102400 2024-05-03T18:47:10.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:10.450+0000 7f5219b75700 1 -- 172.21.15.32:0/4057580654 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f521406b1e0 con 0x7f5214102400 2024-05-03T18:47:10.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:10.450+0000 7f5210ff9700 1 -- 172.21.15.32:0/4057580654 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5204007c60 con 0x7f5214102400 2024-05-03T18:47:10.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:10.450+0000 7f5210ff9700 1 -- 172.21.15.32:0/4057580654 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5204018660 con 0x7f5214102400 2024-05-03T18:47:10.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:10.450+0000 7f5219b75700 1 -- 172.21.15.32:0/4057580654 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f521406b770 con 0x7f5214102400 2024-05-03T18:47:10.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:10.451+0000 7f5219b75700 1 -- 172.21.15.32:0/4057580654 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f521406a190 con 0x7f5214102400 2024-05-03T18:47:10.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:10.452+0000 7f5210ff9700 1 -- 172.21.15.32:0/4057580654 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f5204004020 con 0x7f5214102400 2024-05-03T18:47:10.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:10.453+0000 7f5210ff9700 1 --2- 172.21.15.32:0/4057580654 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f51fc0b2910 0x7f51fc0b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:10.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:10.453+0000 7f5212ffd700 1 --2- 172.21.15.32:0/4057580654 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f51fc0b2910 0x7f51fc0b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:10.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:10.453+0000 7f5210ff9700 1 -- 172.21.15.32:0/4057580654 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8356+0+0 (secure 0 0 0) 0x7f52040e2620 con 0x7f5214102400 2024-05-03T18:47:10.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:10.454+0000 7f5212ffd700 1 --2- 172.21.15.32:0/4057580654 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f51fc0b2910 0x7f51fc0b4dd0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f52080038c0 tx=0x7f5208005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:47:10.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:10.457+0000 7f5210ff9700 1 -- 172.21.15.32:0/4057580654 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f5204053d70 con 0x7f5214102400 2024-05-03T18:47:10.681 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:10.680+0000 7f5219b75700 1 -- 172.21.15.32:0/4057580654 --> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f521406ba30 con 0x7f51fc0b2910 2024-05-03T18:47:10.695 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:10.694+0000 7f5210ff9700 1 -- 172.21.15.32:0/4057580654 <== mgr.14152 v2:172.21.15.32:6800/1796547904 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4123 (secure 0 0 0) 0x7f521406ba30 con 0x7f51fc0b2910 2024-05-03T18:47:10.696 INFO:teuthology.orchestra.run.smithi032.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-03T18:47:10.696 INFO:teuthology.orchestra.run.smithi032.stdout:mds.a smithi032 running (2m) 7s ago 2m 14.4M - 19.0.0-3450-g9a047fe7 ee4d84620249 3a5ed69a8302 2024-05-03T18:47:10.696 INFO:teuthology.orchestra.run.smithi032.stdout:mds.b smithi045 running (2m) 10s ago 2m 21.1M - 19.0.0-3450-g9a047fe7 ee4d84620249 f3935e6c79c2 2024-05-03T18:47:10.696 INFO:teuthology.orchestra.run.smithi032.stdout:mds.c smithi113 running (2m) 10s ago 2m 23.0M - 19.0.0-3450-g9a047fe7 ee4d84620249 f3eae32eb8de 2024-05-03T18:47:10.696 INFO:teuthology.orchestra.run.smithi032.stdout:mds.d smithi032 running (2m) 7s ago 2m 14.5M - 19.0.0-3450-g9a047fe7 ee4d84620249 513c8de5cd19 2024-05-03T18:47:10.696 INFO:teuthology.orchestra.run.smithi032.stdout:mds.e smithi045 running (2m) 10s ago 2m 16.7M - 19.0.0-3450-g9a047fe7 ee4d84620249 5eec80064e6e 2024-05-03T18:47:10.696 INFO:teuthology.orchestra.run.smithi032.stdout:mds.f smithi113 running (2m) 10s ago 2m 20.0M - 19.0.0-3450-g9a047fe7 ee4d84620249 33c4af011e7d 2024-05-03T18:47:10.696 INFO:teuthology.orchestra.run.smithi032.stdout:mds.g smithi032 running (2m) 7s ago 2m 14.6M - 19.0.0-3450-g9a047fe7 ee4d84620249 43d6b3ac5db5 2024-05-03T18:47:10.696 INFO:teuthology.orchestra.run.smithi032.stdout:mds.h smithi045 running (2m) 10s ago 2m 18.7M - 19.0.0-3450-g9a047fe7 ee4d84620249 7872d57d7bb3 2024-05-03T18:47:10.696 INFO:teuthology.orchestra.run.smithi032.stdout:mds.i smithi113 running (2m) 10s ago 2m 26.3M - 19.0.0-3450-g9a047fe7 ee4d84620249 e79590a06bad 2024-05-03T18:47:10.696 INFO:teuthology.orchestra.run.smithi032.stdout:mds.j smithi032 running (3m) 7s ago 3m 14.3M - 19.0.0-3450-g9a047fe7 ee4d84620249 8bfc26b18030 2024-05-03T18:47:10.696 INFO:teuthology.orchestra.run.smithi032.stdout:mds.k smithi045 running (3m) 10s ago 3m 13.9M - 19.0.0-3450-g9a047fe7 ee4d84620249 01184527005b 2024-05-03T18:47:10.697 INFO:teuthology.orchestra.run.smithi032.stdout:mds.l smithi113 running (3m) 10s ago 3m 21.3M - 19.0.0-3450-g9a047fe7 ee4d84620249 4f9cf271e240 2024-05-03T18:47:10.697 INFO:teuthology.orchestra.run.smithi032.stdout:mgr.x smithi032 *:9283,8765 running (10m) 7s ago 10m 529M - 19.0.0-3450-g9a047fe7 ee4d84620249 bc042bf2e9b3 2024-05-03T18:47:10.697 INFO:teuthology.orchestra.run.smithi032.stdout:mgr.y smithi045 *:8443,8765 running (8m) 10s ago 8m 467M - 19.0.0-3450-g9a047fe7 ee4d84620249 57afcca919d6 2024-05-03T18:47:10.697 INFO:teuthology.orchestra.run.smithi032.stdout:mgr.z smithi113 *:8443,8765 running (8m) 10s ago 8m 468M - 19.0.0-3450-g9a047fe7 ee4d84620249 5dbb8da28872 2024-05-03T18:47:10.697 INFO:teuthology.orchestra.run.smithi032.stdout:mon.a smithi032 running (10m) 7s ago 10m 54.3M 2048M 19.0.0-3450-g9a047fe7 ee4d84620249 b018994d5361 2024-05-03T18:47:10.697 INFO:teuthology.orchestra.run.smithi032.stdout:mon.b smithi045 running (8m) 10s ago 8m 40.4M 2048M 19.0.0-3450-g9a047fe7 ee4d84620249 8d050076af56 2024-05-03T18:47:10.697 INFO:teuthology.orchestra.run.smithi032.stdout:mon.c smithi113 running (8m) 10s ago 8m 42.6M 2048M 19.0.0-3450-g9a047fe7 ee4d84620249 c64669fc044e 2024-05-03T18:47:10.697 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0 smithi032 running (7m) 7s ago 7m 55.4M 4273M 19.0.0-3450-g9a047fe7 ee4d84620249 f20a1672f110 2024-05-03T18:47:10.697 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1 smithi045 running (7m) 10s ago 7m 48.5M 4273M 19.0.0-3450-g9a047fe7 ee4d84620249 94b5ca8dfd01 2024-05-03T18:47:10.697 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2 smithi113 running (7m) 10s ago 7m 51.3M 4272M 19.0.0-3450-g9a047fe7 ee4d84620249 20f0cd67f620 2024-05-03T18:47:10.697 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3 smithi032 running (6m) 7s ago 6m 53.5M 4273M 19.0.0-3450-g9a047fe7 ee4d84620249 fdf2be52179d 2024-05-03T18:47:10.697 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4 smithi045 running (6m) 10s ago 6m 48.7M 4273M 19.0.0-3450-g9a047fe7 ee4d84620249 d1a649f52549 2024-05-03T18:47:10.697 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5 smithi113 running (5m) 10s ago 5m 57.9M 4272M 19.0.0-3450-g9a047fe7 ee4d84620249 ac8bbef59b6e 2024-05-03T18:47:10.697 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6 smithi032 running (5m) 7s ago 5m 45.5M 4273M 19.0.0-3450-g9a047fe7 ee4d84620249 56afac64d3ed 2024-05-03T18:47:10.697 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7 smithi045 running (5m) 10s ago 5m 54.7M 4273M 19.0.0-3450-g9a047fe7 ee4d84620249 f9a505103f4f 2024-05-03T18:47:10.697 INFO:teuthology.orchestra.run.smithi032.stdout:osd.8 smithi113 running (4m) 10s ago 4m 46.3M 4272M 19.0.0-3450-g9a047fe7 ee4d84620249 1e4930a84744 2024-05-03T18:47:10.698 INFO:teuthology.orchestra.run.smithi032.stdout:osd.9 smithi032 running (4m) 7s ago 4m 44.8M 4273M 19.0.0-3450-g9a047fe7 ee4d84620249 0649800c937e 2024-05-03T18:47:10.698 INFO:teuthology.orchestra.run.smithi032.stdout:osd.10 smithi045 running (3m) 10s ago 3m 47.6M 4273M 19.0.0-3450-g9a047fe7 ee4d84620249 81e9e188cd80 2024-05-03T18:47:10.698 INFO:teuthology.orchestra.run.smithi032.stdout:osd.11 smithi113 running (3m) 10s ago 3m 46.3M 4272M 19.0.0-3450-g9a047fe7 ee4d84620249 76ea97c5034c 2024-05-03T18:47:10.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:10.697+0000 7f5219b75700 1 -- 172.21.15.32:0/4057580654 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f51fc0b2910 msgr2=0x7f51fc0b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:10.698 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:10.697+0000 7f5219b75700 1 --2- 172.21.15.32:0/4057580654 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f51fc0b2910 0x7f51fc0b4dd0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f52080038c0 tx=0x7f5208005fd0 comp rx=0 tx=0).stop 2024-05-03T18:47:10.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:10.697+0000 7f5219b75700 1 -- 172.21.15.32:0/4057580654 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5214102400 msgr2=0x7f52141984f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:10.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:10.697+0000 7f5219b75700 1 --2- 172.21.15.32:0/4057580654 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5214102400 0x7f52141984f0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f520400c9c0 tx=0x7f520400cd80 comp rx=0 tx=0).stop 2024-05-03T18:47:10.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:10.697+0000 7f5219b75700 1 -- 172.21.15.32:0/4057580654 shutdown_connections 2024-05-03T18:47:10.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:10.697+0000 7f5219b75700 1 --2- 172.21.15.32:0/4057580654 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f51fc0b2910 0x7f51fc0b4dd0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:10.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:10.697+0000 7f5219b75700 1 --2- 172.21.15.32:0/4057580654 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5214103fa0 0x7f521419fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:10.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:10.697+0000 7f5219b75700 1 --2- 172.21.15.32:0/4057580654 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5214102400 0x7f52141984f0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:10.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:10.697+0000 7f5219b75700 1 --2- 172.21.15.32:0/4057580654 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5214103600 0x7f5214198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:10.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:10.697+0000 7f5219b75700 1 -- 172.21.15.32:0/4057580654 >> 172.21.15.32:0/4057580654 conn(0x7f52140fd9a0 msgr2=0x7f5214106a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:47:10.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:10.697+0000 7f5219b75700 1 -- 172.21.15.32:0/4057580654 shutdown_connections 2024-05-03T18:47:10.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:10.697+0000 7f5219b75700 1 -- 172.21.15.32:0/4057580654 wait complete. 2024-05-03T18:47:11.351 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f500455e-097b-11ef-bc97-c7b262605968 -- bash -c 'ceph orch ls' 2024-05-03T18:47:11.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:47:11 smithi113 ceph-mon[33540]: pgmap v354: 97 pgs: 97 active+clean; 631 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:47:11.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:47:11 smithi032 ceph-mon[29763]: pgmap v354: 97 pgs: 97 active+clean; 631 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:47:11.602 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:47:11.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:47:11 smithi045 ceph-mon[33299]: pgmap v354: 97 pgs: 97 active+clean; 631 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:47:12.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:47:12 smithi113 ceph-mon[33540]: from='client.15030 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:47:12.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:47:12 smithi032 ceph-mon[29763]: from='client.15030 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:47:12.747 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:47:12 smithi045 ceph-mon[33299]: from='client.15030 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:47:13.202 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:13.200+0000 7fedc1e96700 1 -- 172.21.15.32:0/1770447536 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fedbc104720 msgr2=0x7fedbc104b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:13.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:13.200+0000 7fedc1e96700 1 --2- 172.21.15.32:0/1770447536 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fedbc104720 0x7fedbc104b80 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fedb00099c0 tx=0x7fedb0009cd0 comp rx=0 tx=0).stop 2024-05-03T18:47:13.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:13.200+0000 7fedc1e96700 1 -- 172.21.15.32:0/1770447536 shutdown_connections 2024-05-03T18:47:13.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:13.200+0000 7fedc1e96700 1 --2- 172.21.15.32:0/1770447536 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fedbc1050c0 0x7fedbc10b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:13.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:13.200+0000 7fedc1e96700 1 --2- 172.21.15.32:0/1770447536 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fedbc103520 0x7fedbc103940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:13.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:13.200+0000 7fedc1e96700 1 --2- 172.21.15.32:0/1770447536 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fedbc104720 0x7fedbc104b80 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:13.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:13.200+0000 7fedc1e96700 1 -- 172.21.15.32:0/1770447536 >> 172.21.15.32:0/1770447536 conn(0x7fedbc0feac0 msgr2=0x7fedbc100f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:47:13.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:13.201+0000 7fedc1e96700 1 -- 172.21.15.32:0/1770447536 shutdown_connections 2024-05-03T18:47:13.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:13.201+0000 7fedc1e96700 1 -- 172.21.15.32:0/1770447536 wait complete. 2024-05-03T18:47:13.203 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:13.202+0000 7fedc1e96700 1 Processor -- start 2024-05-03T18:47:13.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:13.202+0000 7fedc1e96700 1 -- start start 2024-05-03T18:47:13.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:13.202+0000 7fedc1e96700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fedbc103520 0x7fedbc199600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:13.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:13.202+0000 7fedc1e96700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fedbc104720 0x7fedbc199b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:13.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:13.202+0000 7fedc1e96700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fedbc1050c0 0x7fedbc1a0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:13.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:13.202+0000 7fedc1e96700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fedbc10efc0 con 0x7fedbc103520 2024-05-03T18:47:13.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:13.202+0000 7fedc1e96700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fedbc10ee40 con 0x7fedbc1050c0 2024-05-03T18:47:13.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:13.202+0000 7fedc1e96700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fedbc10f140 con 0x7fedbc104720 2024-05-03T18:47:13.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:13.202+0000 7fedbb7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fedbc103520 0x7fedbc199600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:13.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:13.202+0000 7fedbb7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fedbc103520 0x7fedbc199600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57552/0 (socket says 172.21.15.32:57552) 2024-05-03T18:47:13.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:13.202+0000 7fedbb7fe700 1 -- 172.21.15.32:0/719516024 learned_addr learned my addr 172.21.15.32:0/719516024 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:47:13.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:13.202+0000 7fedbaffd700 1 --2- 172.21.15.32:0/719516024 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fedbc104720 0x7fedbc199b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:13.205 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:13.202+0000 7fedbbfff700 1 --2- 172.21.15.32:0/719516024 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fedbc1050c0 0x7fedbc1a0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:13.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:13.203+0000 7fedbb7fe700 1 -- 172.21.15.32:0/719516024 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fedbc104720 msgr2=0x7fedbc199b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:13.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:13.203+0000 7fedbb7fe700 1 --2- 172.21.15.32:0/719516024 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fedbc104720 0x7fedbc199b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:13.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:13.203+0000 7fedbb7fe700 1 -- 172.21.15.32:0/719516024 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fedbc1050c0 msgr2=0x7fedbc1a0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:13.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:13.203+0000 7fedbb7fe700 1 --2- 172.21.15.32:0/719516024 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fedbc1050c0 0x7fedbc1a0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:13.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:13.203+0000 7fedbb7fe700 1 -- 172.21.15.32:0/719516024 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedbc1a1260 con 0x7fedbc103520 2024-05-03T18:47:13.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:13.203+0000 7fedbb7fe700 1 --2- 172.21.15.32:0/719516024 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fedbc103520 0x7fedbc199600 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fedac00d9b0 tx=0x7fedac00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:47:13.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:13.203+0000 7fedb8ff9700 1 -- 172.21.15.32:0/719516024 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedac017070 con 0x7fedbc103520 2024-05-03T18:47:13.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:13.203+0000 7fedc1e96700 1 -- 172.21.15.32:0/719516024 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedbc1a1550 con 0x7fedbc103520 2024-05-03T18:47:13.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:13.203+0000 7fedb8ff9700 1 -- 172.21.15.32:0/719516024 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fedac009de0 con 0x7fedbc103520 2024-05-03T18:47:13.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:13.203+0000 7fedb8ff9700 1 -- 172.21.15.32:0/719516024 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedac00f3c0 con 0x7fedbc103520 2024-05-03T18:47:13.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:13.203+0000 7fedc1e96700 1 -- 172.21.15.32:0/719516024 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fedbc1a1a40 con 0x7fedbc103520 2024-05-03T18:47:13.206 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:13.204+0000 7fedc1e96700 1 -- 172.21.15.32:0/719516024 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fedbc06a190 con 0x7fedbc103520 2024-05-03T18:47:13.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:13.206+0000 7fedb8ff9700 1 -- 172.21.15.32:0/719516024 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fedac009930 con 0x7fedbc103520 2024-05-03T18:47:13.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:13.207+0000 7fedb8ff9700 1 --2- 172.21.15.32:0/719516024 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7feda40b2910 0x7feda40b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:13.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:13.207+0000 7fedbaffd700 1 --2- 172.21.15.32:0/719516024 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7feda40b2910 0x7feda40b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:13.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:13.207+0000 7fedb8ff9700 1 -- 172.21.15.32:0/719516024 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8356+0+0 (secure 0 0 0) 0x7fedac0e1ea0 con 0x7fedbc103520 2024-05-03T18:47:13.208 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:13.208+0000 7fedbaffd700 1 --2- 172.21.15.32:0/719516024 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7feda40b2910 0x7feda40b4dd0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fedb0009990 tx=0x7fedb000b870 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:47:13.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:13.211+0000 7fedb8ff9700 1 -- 172.21.15.32:0/719516024 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fedac0a9980 con 0x7fedbc103520 2024-05-03T18:47:13.439 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:13.438+0000 7fedc1e96700 1 -- 172.21.15.32:0/719516024 --> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7fedbc0638e0 con 0x7feda40b2910 2024-05-03T18:47:13.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:13.444+0000 7fedb8ff9700 1 -- 172.21.15.32:0/719516024 <== mgr.14152 v2:172.21.15.32:6800/1796547904 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7fedbc0638e0 con 0x7feda40b2910 2024-05-03T18:47:13.446 INFO:teuthology.orchestra.run.smithi032.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-05-03T18:47:13.446 INFO:teuthology.orchestra.run.smithi032.stdout:mds.all 12/12 13s ago 3m smithi032=a;smithi032=d;smithi032=g;smithi032=j;smithi045=b;smithi045=e;smithi045=h;smithi045=k;smithi113=c;smithi113=f;smithi113=i;smithi113=l;count:12 2024-05-03T18:47:13.446 INFO:teuthology.orchestra.run.smithi032.stdout:mgr 3/3 13s ago 8m smithi032=x;smithi045=y;smithi113=z;count:3 2024-05-03T18:47:13.446 INFO:teuthology.orchestra.run.smithi032.stdout:mon 3/3 13s ago 8m smithi032:172.21.15.32=a;smithi045:172.21.15.45=b;smithi113:172.21.15.113=c;count:3 2024-05-03T18:47:13.446 INFO:teuthology.orchestra.run.smithi032.stdout:osd 12 13s ago - 2024-05-03T18:47:13.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:13.446+0000 7fedc1e96700 1 -- 172.21.15.32:0/719516024 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7feda40b2910 msgr2=0x7feda40b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:13.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:13.446+0000 7fedc1e96700 1 --2- 172.21.15.32:0/719516024 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7feda40b2910 0x7feda40b4dd0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fedb0009990 tx=0x7fedb000b870 comp rx=0 tx=0).stop 2024-05-03T18:47:13.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:13.447+0000 7fedc1e96700 1 -- 172.21.15.32:0/719516024 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fedbc103520 msgr2=0x7fedbc199600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:13.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:13.447+0000 7fedc1e96700 1 --2- 172.21.15.32:0/719516024 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fedbc103520 0x7fedbc199600 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fedac00d9b0 tx=0x7fedac00dcc0 comp rx=0 tx=0).stop 2024-05-03T18:47:13.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:13.447+0000 7fedc1e96700 1 -- 172.21.15.32:0/719516024 shutdown_connections 2024-05-03T18:47:13.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:13.447+0000 7fedc1e96700 1 --2- 172.21.15.32:0/719516024 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7feda40b2910 0x7feda40b4dd0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:13.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:13.447+0000 7fedc1e96700 1 --2- 172.21.15.32:0/719516024 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fedbc1050c0 0x7fedbc1a0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:13.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:13.447+0000 7fedc1e96700 1 --2- 172.21.15.32:0/719516024 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fedbc103520 0x7fedbc199600 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:13.448 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:13.447+0000 7fedc1e96700 1 --2- 172.21.15.32:0/719516024 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fedbc104720 0x7fedbc199b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:13.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:13.447+0000 7fedc1e96700 1 -- 172.21.15.32:0/719516024 >> 172.21.15.32:0/719516024 conn(0x7fedbc0feac0 msgr2=0x7fedbc107a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:47:13.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:13.447+0000 7fedc1e96700 1 -- 172.21.15.32:0/719516024 shutdown_connections 2024-05-03T18:47:13.449 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:13.447+0000 7fedc1e96700 1 -- 172.21.15.32:0/719516024 wait complete. 2024-05-03T18:47:13.459 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:47:13 smithi032 ceph-mon[29763]: pgmap v355: 97 pgs: 97 active+clean; 631 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:47:13.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:47:13 smithi113 ceph-mon[33540]: pgmap v355: 97 pgs: 97 active+clean; 631 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:47:13.747 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:47:13 smithi045 ceph-mon[33299]: pgmap v355: 97 pgs: 97 active+clean; 631 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:47:14.065 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f500455e-097b-11ef-bc97-c7b262605968 -- bash -c 'ceph orch host ls' 2024-05-03T18:47:14.317 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:47:14.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:47:14 smithi032 ceph-mon[29763]: from='client.15036 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:47:14.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:47:14 smithi113 ceph-mon[33540]: from='client.15036 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:47:14.747 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:47:14 smithi045 ceph-mon[33299]: from='client.15036 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:47:15.408 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:47:15 smithi032 ceph-mon[29763]: pgmap v356: 97 pgs: 97 active+clean; 631 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:47:15.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:47:15 smithi113 ceph-mon[33540]: pgmap v356: 97 pgs: 97 active+clean; 631 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:47:15.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:47:15 smithi045 ceph-mon[33299]: pgmap v356: 97 pgs: 97 active+clean; 631 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:47:15.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:15.951+0000 7f1236162700 1 -- 172.21.15.32:0/743439496 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1230103fe0 msgr2=0x7f123010a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:15.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:15.951+0000 7f1236162700 1 --2- 172.21.15.32:0/743439496 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1230103fe0 0x7f123010a890 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f122800b0d0 tx=0x7f122800b3e0 comp rx=0 tx=0).stop 2024-05-03T18:47:15.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:15.951+0000 7f1236162700 1 -- 172.21.15.32:0/743439496 shutdown_connections 2024-05-03T18:47:15.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:15.951+0000 7f1236162700 1 --2- 172.21.15.32:0/743439496 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1230103fe0 0x7f123010a890 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:15.954 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:15.951+0000 7f1236162700 1 --2- 172.21.15.32:0/743439496 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1230102440 0x7f1230102860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:15.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:15.951+0000 7f1236162700 1 --2- 172.21.15.32:0/743439496 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1230103640 0x7f1230103aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:15.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:15.951+0000 7f1236162700 1 -- 172.21.15.32:0/743439496 >> 172.21.15.32:0/743439496 conn(0x7f12300fda00 msgr2=0x7f12300ffe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:47:15.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:15.951+0000 7f1236162700 1 -- 172.21.15.32:0/743439496 shutdown_connections 2024-05-03T18:47:15.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:15.951+0000 7f1236162700 1 -- 172.21.15.32:0/743439496 wait complete. 2024-05-03T18:47:15.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:15.952+0000 7f1236162700 1 Processor -- start 2024-05-03T18:47:15.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:15.952+0000 7f1236162700 1 -- start start 2024-05-03T18:47:15.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:15.952+0000 7f1236162700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1230102440 0x7f1230199530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:15.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:15.952+0000 7f1236162700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1230103640 0x7f1230199a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:15.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:15.952+0000 7f1236162700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1230103fe0 0x7f12301a0af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:15.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:15.952+0000 7f1236162700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f123010dee0 con 0x7f1230103640 2024-05-03T18:47:15.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:15.952+0000 7f1236162700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f123010dd60 con 0x7f1230102440 2024-05-03T18:47:15.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:15.952+0000 7f1236162700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f123010e060 con 0x7f1230103fe0 2024-05-03T18:47:15.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:15.952+0000 7f122effd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1230103640 0x7f1230199a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:15.955 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:15.952+0000 7f122effd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1230103640 0x7f1230199a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46490/0 (socket says 172.21.15.32:46490) 2024-05-03T18:47:15.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:15.952+0000 7f122effd700 1 -- 172.21.15.32:0/17768509 learned_addr learned my addr 172.21.15.32:0/17768509 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:47:15.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:15.952+0000 7f122f7fe700 1 --2- 172.21.15.32:0/17768509 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1230102440 0x7f1230199530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:15.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:15.952+0000 7f122ffff700 1 --2- 172.21.15.32:0/17768509 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1230103fe0 0x7f12301a0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:15.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:15.952+0000 7f122effd700 1 -- 172.21.15.32:0/17768509 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1230103fe0 msgr2=0x7f12301a0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:15.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:15.952+0000 7f122effd700 1 --2- 172.21.15.32:0/17768509 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1230103fe0 0x7f12301a0af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:15.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:15.952+0000 7f122effd700 1 -- 172.21.15.32:0/17768509 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1230102440 msgr2=0x7f1230199530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:15.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:15.952+0000 7f122effd700 1 --2- 172.21.15.32:0/17768509 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1230102440 0x7f1230199530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:15.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:15.953+0000 7f122effd700 1 -- 172.21.15.32:0/17768509 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12301a1190 con 0x7f1230103640 2024-05-03T18:47:15.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:15.953+0000 7f122ffff700 1 --2- 172.21.15.32:0/17768509 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1230103fe0 0x7f12301a0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:47:15.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:15.953+0000 7f122effd700 1 --2- 172.21.15.32:0/17768509 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1230103640 0x7f1230199a70 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f122400d9b0 tx=0x7f122400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:47:15.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:15.953+0000 7f122cff9700 1 -- 172.21.15.32:0/17768509 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1224017070 con 0x7f1230103640 2024-05-03T18:47:15.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:15.953+0000 7f1236162700 1 -- 172.21.15.32:0/17768509 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12301a1480 con 0x7f1230103640 2024-05-03T18:47:15.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:15.953+0000 7f122cff9700 1 -- 172.21.15.32:0/17768509 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1224005020 con 0x7f1230103640 2024-05-03T18:47:15.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:15.953+0000 7f122cff9700 1 -- 172.21.15.32:0/17768509 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1224009b50 con 0x7f1230103640 2024-05-03T18:47:15.956 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:15.953+0000 7f1236162700 1 -- 172.21.15.32:0/17768509 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12301a1970 con 0x7f1230103640 2024-05-03T18:47:15.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:15.954+0000 7f1236162700 1 -- 172.21.15.32:0/17768509 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f123006a190 con 0x7f1230103640 2024-05-03T18:47:15.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:15.955+0000 7f122cff9700 1 -- 172.21.15.32:0/17768509 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f12240051d0 con 0x7f1230103640 2024-05-03T18:47:15.957 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:15.956+0000 7f122cff9700 1 --2- 172.21.15.32:0/17768509 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f12180b2910 0x7f12180b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:15.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:15.956+0000 7f122f7fe700 1 --2- 172.21.15.32:0/17768509 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f12180b2910 0x7f12180b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:15.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:15.956+0000 7f122cff9700 1 -- 172.21.15.32:0/17768509 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8356+0+0 (secure 0 0 0) 0x7f12240e2380 con 0x7f1230103640 2024-05-03T18:47:15.958 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:15.956+0000 7f122f7fe700 1 --2- 172.21.15.32:0/17768509 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f12180b2910 0x7f12180b4dd0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f12301034a0 tx=0x7f122000a3b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:47:15.962 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:15.961+0000 7f122cff9700 1 -- 172.21.15.32:0/17768509 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f1224020020 con 0x7f1230103640 2024-05-03T18:47:16.184 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:16.183+0000 7f1236162700 1 -- 172.21.15.32:0/17768509 --> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f12300ffab0 con 0x7f12180b2910 2024-05-03T18:47:16.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:16.184+0000 7f122cff9700 1 -- 172.21.15.32:0/17768509 <== mgr.14152 v2:172.21.15.32:6800/1796547904 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f12300ffab0 con 0x7f12180b2910 2024-05-03T18:47:16.185 INFO:teuthology.orchestra.run.smithi032.stdout:HOST ADDR LABELS STATUS 2024-05-03T18:47:16.185 INFO:teuthology.orchestra.run.smithi032.stdout:smithi032 172.21.15.32 2024-05-03T18:47:16.186 INFO:teuthology.orchestra.run.smithi032.stdout:smithi045 172.21.15.45 2024-05-03T18:47:16.186 INFO:teuthology.orchestra.run.smithi032.stdout:smithi113 172.21.15.113 2024-05-03T18:47:16.186 INFO:teuthology.orchestra.run.smithi032.stdout:3 hosts in cluster 2024-05-03T18:47:16.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:16.187+0000 7f1236162700 1 -- 172.21.15.32:0/17768509 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f12180b2910 msgr2=0x7f12180b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:16.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:16.187+0000 7f1236162700 1 --2- 172.21.15.32:0/17768509 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f12180b2910 0x7f12180b4dd0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f12301034a0 tx=0x7f122000a3b0 comp rx=0 tx=0).stop 2024-05-03T18:47:16.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:16.187+0000 7f1236162700 1 -- 172.21.15.32:0/17768509 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1230103640 msgr2=0x7f1230199a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:16.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:16.187+0000 7f1236162700 1 --2- 172.21.15.32:0/17768509 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1230103640 0x7f1230199a70 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f122400d9b0 tx=0x7f122400dcc0 comp rx=0 tx=0).stop 2024-05-03T18:47:16.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:16.187+0000 7f1236162700 1 -- 172.21.15.32:0/17768509 shutdown_connections 2024-05-03T18:47:16.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:16.187+0000 7f1236162700 1 --2- 172.21.15.32:0/17768509 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f12180b2910 0x7f12180b4dd0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:16.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:16.187+0000 7f1236162700 1 --2- 172.21.15.32:0/17768509 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1230103fe0 0x7f12301a0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:16.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:16.187+0000 7f1236162700 1 --2- 172.21.15.32:0/17768509 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1230102440 0x7f1230199530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:16.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:16.187+0000 7f1236162700 1 --2- 172.21.15.32:0/17768509 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1230103640 0x7f1230199a70 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:16.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:16.187+0000 7f1236162700 1 -- 172.21.15.32:0/17768509 >> 172.21.15.32:0/17768509 conn(0x7f12300fda00 msgr2=0x7f12300ff3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:47:16.188 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:16.187+0000 7f1236162700 1 -- 172.21.15.32:0/17768509 shutdown_connections 2024-05-03T18:47:16.189 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:16.187+0000 7f1236162700 1 -- 172.21.15.32:0/17768509 wait complete. 2024-05-03T18:47:16.829 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f500455e-097b-11ef-bc97-c7b262605968 -- bash -c 'ceph orch device ls' 2024-05-03T18:47:17.077 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:47:17.479 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:47:17 smithi032 ceph-mon[29763]: pgmap v357: 97 pgs: 97 active+clean; 631 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:47:17.479 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:47:17 smithi032 ceph-mon[29763]: from='client.15042 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:47:17.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:47:17 smithi113 ceph-mon[33540]: pgmap v357: 97 pgs: 97 active+clean; 631 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:47:17.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:47:17 smithi113 ceph-mon[33540]: from='client.15042 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:47:17.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:47:17 smithi045 ceph-mon[33299]: pgmap v357: 97 pgs: 97 active+clean; 631 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:47:17.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:47:17 smithi045 ceph-mon[33299]: from='client.15042 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:47:18.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:18.768+0000 7f3b47e55700 1 -- 172.21.15.32:0/3400088194 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3b40104720 msgr2=0x7f3b40104b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:18.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:18.768+0000 7f3b47e55700 1 --2- 172.21.15.32:0/3400088194 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3b40104720 0x7f3b40104b80 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f3b340099c0 tx=0x7f3b34009cd0 comp rx=0 tx=0).stop 2024-05-03T18:47:18.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:18.768+0000 7f3b47e55700 1 -- 172.21.15.32:0/3400088194 shutdown_connections 2024-05-03T18:47:18.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:18.768+0000 7f3b47e55700 1 --2- 172.21.15.32:0/3400088194 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3b401050c0 0x7f3b4010b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:18.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:18.768+0000 7f3b47e55700 1 --2- 172.21.15.32:0/3400088194 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3b40103520 0x7f3b40103940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:18.771 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:18.768+0000 7f3b47e55700 1 --2- 172.21.15.32:0/3400088194 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3b40104720 0x7f3b40104b80 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:18.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:18.768+0000 7f3b47e55700 1 -- 172.21.15.32:0/3400088194 >> 172.21.15.32:0/3400088194 conn(0x7f3b400feac0 msgr2=0x7f3b40100f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:47:18.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:18.768+0000 7f3b47e55700 1 -- 172.21.15.32:0/3400088194 shutdown_connections 2024-05-03T18:47:18.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:18.768+0000 7f3b47e55700 1 -- 172.21.15.32:0/3400088194 wait complete. 2024-05-03T18:47:18.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:18.769+0000 7f3b47e55700 1 Processor -- start 2024-05-03T18:47:18.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:18.769+0000 7f3b47e55700 1 -- start start 2024-05-03T18:47:18.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:18.769+0000 7f3b47e55700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3b40103520 0x7f3b401996f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:18.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:18.769+0000 7f3b47e55700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3b40104720 0x7f3b40199c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:18.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:18.769+0000 7f3b47e55700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3b401050c0 0x7f3b401a0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:18.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:18.769+0000 7f3b47e55700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f3b4010f080 con 0x7f3b40103520 2024-05-03T18:47:18.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:18.769+0000 7f3b47e55700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3b4010ef00 con 0x7f3b401050c0 2024-05-03T18:47:18.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:18.769+0000 7f3b47e55700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f3b4010f200 con 0x7f3b40104720 2024-05-03T18:47:18.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:18.769+0000 7f3b45bf1700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3b40103520 0x7f3b401996f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:18.772 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:18.769+0000 7f3b45bf1700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3b40103520 0x7f3b401996f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46502/0 (socket says 172.21.15.32:46502) 2024-05-03T18:47:18.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:18.769+0000 7f3b45bf1700 1 -- 172.21.15.32:0/1638198073 learned_addr learned my addr 172.21.15.32:0/1638198073 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:47:18.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:18.769+0000 7f3b453f0700 1 --2- 172.21.15.32:0/1638198073 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3b40104720 0x7f3b40199c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:18.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:18.769+0000 7f3b463f2700 1 --2- 172.21.15.32:0/1638198073 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3b401050c0 0x7f3b401a0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:18.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:18.769+0000 7f3b45bf1700 1 -- 172.21.15.32:0/1638198073 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3b40104720 msgr2=0x7f3b40199c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:18.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:18.769+0000 7f3b45bf1700 1 --2- 172.21.15.32:0/1638198073 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3b40104720 0x7f3b40199c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:18.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:18.769+0000 7f3b45bf1700 1 -- 172.21.15.32:0/1638198073 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3b401050c0 msgr2=0x7f3b401a0cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:18.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:18.769+0000 7f3b45bf1700 1 --2- 172.21.15.32:0/1638198073 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3b401050c0 0x7f3b401a0cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:18.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:18.769+0000 7f3b45bf1700 1 -- 172.21.15.32:0/1638198073 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b401a1350 con 0x7f3b40103520 2024-05-03T18:47:18.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:18.770+0000 7f3b45bf1700 1 --2- 172.21.15.32:0/1638198073 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3b40103520 0x7f3b401996f0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f3b3000add0 tx=0x7f3b3000c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:47:18.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:18.770+0000 7f3b3effd700 1 -- 172.21.15.32:0/1638198073 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b30017070 con 0x7f3b40103520 2024-05-03T18:47:18.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:18.770+0000 7f3b3effd700 1 -- 172.21.15.32:0/1638198073 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b30004bb0 con 0x7f3b40103520 2024-05-03T18:47:18.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:18.770+0000 7f3b47e55700 1 -- 172.21.15.32:0/1638198073 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b401a1640 con 0x7f3b40103520 2024-05-03T18:47:18.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:18.770+0000 7f3b3effd700 1 -- 172.21.15.32:0/1638198073 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b3000cd70 con 0x7f3b40103520 2024-05-03T18:47:18.773 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:18.770+0000 7f3b47e55700 1 -- 172.21.15.32:0/1638198073 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b401a1b80 con 0x7f3b40103520 2024-05-03T18:47:18.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:18.771+0000 7f3b47e55700 1 -- 172.21.15.32:0/1638198073 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b4006a190 con 0x7f3b40103520 2024-05-03T18:47:18.774 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:18.772+0000 7f3b3effd700 1 -- 172.21.15.32:0/1638198073 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f3b3001a070 con 0x7f3b40103520 2024-05-03T18:47:18.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:18.773+0000 7f3b3effd700 1 --2- 172.21.15.32:0/1638198073 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f3b280b2990 0x7f3b280b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:18.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:18.773+0000 7f3b453f0700 1 --2- 172.21.15.32:0/1638198073 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f3b280b2990 0x7f3b280b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:18.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:18.773+0000 7f3b3effd700 1 -- 172.21.15.32:0/1638198073 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8356+0+0 (secure 0 0 0) 0x7f3b300e1400 con 0x7f3b40103520 2024-05-03T18:47:18.775 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:18.774+0000 7f3b453f0700 1 --2- 172.21.15.32:0/1638198073 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f3b280b2990 0x7f3b280b4e50 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f3b34006010 tx=0x7f3b34003860 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:47:18.779 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:18.778+0000 7f3b3effd700 1 -- 172.21.15.32:0/1638198073 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f3b300a8ee0 con 0x7f3b40103520 2024-05-03T18:47:19.004 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:19.003+0000 7f3b47e55700 1 -- 172.21.15.32:0/1638198073 --> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f3b400638e0 con 0x7f3b280b2990 2024-05-03T18:47:19.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:19.005+0000 7f3b3effd700 1 -- 172.21.15.32:0/1638198073 <== mgr.14152 v2:172.21.15.32:6800/1796547904 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f3b400638e0 con 0x7f3b280b2990 2024-05-03T18:47:19.006 INFO:teuthology.orchestra.run.smithi032.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-05-03T18:47:19.006 INFO:teuthology.orchestra.run.smithi032.stdout:smithi032 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB747402DN480DGN 447G No 4m ago Has a FileSystem, LVM detected 2024-05-03T18:47:19.006 INFO:teuthology.orchestra.run.smithi032.stdout:smithi045 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB75120098480DGN 447G No 3m ago Has a FileSystem, LVM detected 2024-05-03T18:47:19.006 INFO:teuthology.orchestra.run.smithi032.stdout:smithi113 /dev/nvme0n1 ssd INTEL_SSDPEDMX020T4_CVPD5171000M2P0SGN 1863G No 3m ago Has a FileSystem, LVM detected 2024-05-03T18:47:19.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:19.007+0000 7f3b47e55700 1 -- 172.21.15.32:0/1638198073 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f3b280b2990 msgr2=0x7f3b280b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:19.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:19.007+0000 7f3b47e55700 1 --2- 172.21.15.32:0/1638198073 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f3b280b2990 0x7f3b280b4e50 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f3b34006010 tx=0x7f3b34003860 comp rx=0 tx=0).stop 2024-05-03T18:47:19.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:19.007+0000 7f3b47e55700 1 -- 172.21.15.32:0/1638198073 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3b40103520 msgr2=0x7f3b401996f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:19.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:19.007+0000 7f3b47e55700 1 --2- 172.21.15.32:0/1638198073 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3b40103520 0x7f3b401996f0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f3b3000add0 tx=0x7f3b3000c330 comp rx=0 tx=0).stop 2024-05-03T18:47:19.008 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:19.007+0000 7f3b47e55700 1 -- 172.21.15.32:0/1638198073 shutdown_connections 2024-05-03T18:47:19.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:19.007+0000 7f3b47e55700 1 --2- 172.21.15.32:0/1638198073 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f3b280b2990 0x7f3b280b4e50 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:19.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:19.007+0000 7f3b47e55700 1 --2- 172.21.15.32:0/1638198073 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3b401050c0 0x7f3b401a0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:19.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:19.007+0000 7f3b47e55700 1 --2- 172.21.15.32:0/1638198073 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3b40103520 0x7f3b401996f0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:19.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:19.007+0000 7f3b47e55700 1 --2- 172.21.15.32:0/1638198073 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3b40104720 0x7f3b40199c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:19.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:19.007+0000 7f3b47e55700 1 -- 172.21.15.32:0/1638198073 >> 172.21.15.32:0/1638198073 conn(0x7f3b400feac0 msgr2=0x7f3b40107a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:47:19.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:19.007+0000 7f3b47e55700 1 -- 172.21.15.32:0/1638198073 shutdown_connections 2024-05-03T18:47:19.009 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:19.007+0000 7f3b47e55700 1 -- 172.21.15.32:0/1638198073 wait complete. 2024-05-03T18:47:19.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:47:19 smithi113 ceph-mon[33540]: pgmap v358: 97 pgs: 97 active+clean; 631 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:47:19.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:47:19 smithi032 ceph-mon[29763]: pgmap v358: 97 pgs: 97 active+clean; 631 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:47:19.651 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-03T18:47:19.662 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi032.front.sepia.ceph.com 2024-05-03T18:47:19.663 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f500455e-097b-11ef-bc97-c7b262605968 -- bash -c 'ceph fs dump' 2024-05-03T18:47:19.747 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:47:19 smithi045 ceph-mon[33299]: pgmap v358: 97 pgs: 97 active+clean; 631 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:47:19.910 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:47:20.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:47:20 smithi032 ceph-mon[29763]: from='client.15048 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:47:20.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:47:20 smithi113 ceph-mon[33540]: from='client.15048 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:47:20.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:47:20 smithi045 ceph-mon[33299]: from='client.15048 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:47:21.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:47:21 smithi032 ceph-mon[29763]: pgmap v359: 97 pgs: 97 active+clean; 631 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:47:21.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:47:21 smithi113 ceph-mon[33540]: pgmap v359: 97 pgs: 97 active+clean; 631 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:47:21.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:21.575+0000 7f79a4029700 1 -- 172.21.15.32:0/3784519699 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f799c103520 msgr2=0x7f799c103940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:21.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:21.575+0000 7f79a4029700 1 --2- 172.21.15.32:0/3784519699 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f799c103520 0x7f799c103940 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f79980099c0 tx=0x7f7998009cd0 comp rx=0 tx=0).stop 2024-05-03T18:47:21.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:21.575+0000 7f79a4029700 1 -- 172.21.15.32:0/3784519699 shutdown_connections 2024-05-03T18:47:21.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:21.575+0000 7f79a4029700 1 --2- 172.21.15.32:0/3784519699 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f799c1050c0 0x7f799c10b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:21.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:21.575+0000 7f79a4029700 1 --2- 172.21.15.32:0/3784519699 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f799c103520 0x7f799c103940 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:21.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:21.575+0000 7f79a4029700 1 --2- 172.21.15.32:0/3784519699 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f799c104720 0x7f799c104b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:21.578 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:21.575+0000 7f79a4029700 1 -- 172.21.15.32:0/3784519699 >> 172.21.15.32:0/3784519699 conn(0x7f799c0feac0 msgr2=0x7f799c100f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:47:21.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:21.575+0000 7f79a4029700 1 -- 172.21.15.32:0/3784519699 shutdown_connections 2024-05-03T18:47:21.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:21.575+0000 7f79a4029700 1 -- 172.21.15.32:0/3784519699 wait complete. 2024-05-03T18:47:21.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:21.575+0000 7f79a4029700 1 Processor -- start 2024-05-03T18:47:21.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:21.576+0000 7f79a4029700 1 -- start start 2024-05-03T18:47:21.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:21.576+0000 7f79a4029700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f799c103520 0x7f799c1996f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:21.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:21.576+0000 7f79a4029700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f799c104720 0x7f799c199c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:21.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:21.576+0000 7f79a4029700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f799c1050c0 0x7f799c1a0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:21.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:21.576+0000 7f79a4029700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f799c10f020 con 0x7f799c104720 2024-05-03T18:47:21.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:21.576+0000 7f79a4029700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f799c10eea0 con 0x7f799c1050c0 2024-05-03T18:47:21.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:21.576+0000 7f79a4029700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f799c10f1a0 con 0x7f799c103520 2024-05-03T18:47:21.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:21.576+0000 7f79a15c4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f799c104720 0x7f799c199c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:21.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:21.576+0000 7f79a15c4700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f799c104720 0x7f799c199c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46518/0 (socket says 172.21.15.32:46518) 2024-05-03T18:47:21.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:21.576+0000 7f79a15c4700 1 -- 172.21.15.32:0/3063729054 learned_addr learned my addr 172.21.15.32:0/3063729054 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:47:21.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:21.576+0000 7f79a25c6700 1 --2- 172.21.15.32:0/3063729054 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f799c1050c0 0x7f799c1a0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:21.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:21.576+0000 7f79a1dc5700 1 --2- 172.21.15.32:0/3063729054 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f799c103520 0x7f799c1996f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:21.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:21.576+0000 7f79a15c4700 1 -- 172.21.15.32:0/3063729054 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f799c103520 msgr2=0x7f799c1996f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:21.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:21.576+0000 7f79a15c4700 1 --2- 172.21.15.32:0/3063729054 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f799c103520 0x7f799c1996f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:21.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:21.576+0000 7f79a15c4700 1 -- 172.21.15.32:0/3063729054 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f799c1050c0 msgr2=0x7f799c1a0cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:21.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:21.576+0000 7f79a15c4700 1 --2- 172.21.15.32:0/3063729054 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f799c1050c0 0x7f799c1a0cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:21.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:21.576+0000 7f79a15c4700 1 -- 172.21.15.32:0/3063729054 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f799c1a1350 con 0x7f799c104720 2024-05-03T18:47:21.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:21.576+0000 7f79a25c6700 1 --2- 172.21.15.32:0/3063729054 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f799c1050c0 0x7f799c1a0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:47:21.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:21.576+0000 7f79a15c4700 1 --2- 172.21.15.32:0/3063729054 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f799c104720 0x7f799c199c30 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f798c00c950 tx=0x7f798c00cd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:47:21.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:21.576+0000 7f7992ffd700 1 -- 172.21.15.32:0/3063729054 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f798c007ac0 con 0x7f799c104720 2024-05-03T18:47:21.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:21.577+0000 7f79a4029700 1 -- 172.21.15.32:0/3063729054 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f799c1a1640 con 0x7f799c104720 2024-05-03T18:47:21.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:21.577+0000 7f7992ffd700 1 -- 172.21.15.32:0/3063729054 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f798c004ce0 con 0x7f799c104720 2024-05-03T18:47:21.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:21.577+0000 7f7992ffd700 1 -- 172.21.15.32:0/3063729054 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f798c0055a0 con 0x7f799c104720 2024-05-03T18:47:21.580 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:21.577+0000 7f79a4029700 1 -- 172.21.15.32:0/3063729054 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f799c1a1b80 con 0x7f799c104720 2024-05-03T18:47:21.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:21.577+0000 7f79a4029700 1 -- 172.21.15.32:0/3063729054 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f799c06a190 con 0x7f799c104720 2024-05-03T18:47:21.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:21.579+0000 7f7992ffd700 1 -- 172.21.15.32:0/3063729054 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f798c007c60 con 0x7f799c104720 2024-05-03T18:47:21.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:21.580+0000 7f7992ffd700 1 --2- 172.21.15.32:0/3063729054 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f79880b2910 0x7f79880b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:21.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:21.580+0000 7f79a1dc5700 1 --2- 172.21.15.32:0/3063729054 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f79880b2910 0x7f79880b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:21.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:21.580+0000 7f7992ffd700 1 -- 172.21.15.32:0/3063729054 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8356+0+0 (secure 0 0 0) 0x7f798c0e2490 con 0x7f799c104720 2024-05-03T18:47:21.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:21.580+0000 7f79a1dc5700 1 --2- 172.21.15.32:0/3063729054 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f79880b2910 0x7f79880b4dd0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f79980038c0 tx=0x7f7998000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:47:21.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:21.583+0000 7f7992ffd700 1 -- 172.21.15.32:0/3063729054 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f798c014030 con 0x7f799c104720 2024-05-03T18:47:21.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:47:21 smithi045 ceph-mon[33299]: pgmap v359: 97 pgs: 97 active+clean; 631 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:47:21.847 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:21.846+0000 7f79a4029700 1 -- 172.21.15.32:0/3063729054 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f799c02e540 con 0x7f799c104720 2024-05-03T18:47:21.848 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:21.846+0000 7f7992ffd700 1 -- 172.21.15.32:0/3063729054 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 25 v25) ==== 76+0+3044 (secure 0 0 0) 0x7f798c0a9f70 con 0x7f799c104720 2024-05-03T18:47:21.848 INFO:teuthology.orchestra.run.smithi032.stdout:e25 2024-05-03T18:47:21.848 INFO:teuthology.orchestra.run.smithi032.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-03T18:47:21.848 INFO:teuthology.orchestra.run.smithi032.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-03T18:47:21.848 INFO:teuthology.orchestra.run.smithi032.stdout:legacy client fscid: 1 2024-05-03T18:47:21.848 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-03T18:47:21.848 INFO:teuthology.orchestra.run.smithi032.stdout:Filesystem 'cephfs' (1) 2024-05-03T18:47:21.848 INFO:teuthology.orchestra.run.smithi032.stdout:fs_name cephfs 2024-05-03T18:47:21.848 INFO:teuthology.orchestra.run.smithi032.stdout:epoch 25 2024-05-03T18:47:21.848 INFO:teuthology.orchestra.run.smithi032.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-03T18:47:21.848 INFO:teuthology.orchestra.run.smithi032.stdout:created 2024-05-03T18:44:51.155050+0000 2024-05-03T18:47:21.848 INFO:teuthology.orchestra.run.smithi032.stdout:modified 2024-05-03T18:45:08.404530+0000 2024-05-03T18:47:21.849 INFO:teuthology.orchestra.run.smithi032.stdout:tableserver 0 2024-05-03T18:47:21.849 INFO:teuthology.orchestra.run.smithi032.stdout:root 0 2024-05-03T18:47:21.849 INFO:teuthology.orchestra.run.smithi032.stdout:session_timeout 300 2024-05-03T18:47:21.849 INFO:teuthology.orchestra.run.smithi032.stdout:session_autoclose 300 2024-05-03T18:47:21.849 INFO:teuthology.orchestra.run.smithi032.stdout:max_file_size 1099511627776 2024-05-03T18:47:21.849 INFO:teuthology.orchestra.run.smithi032.stdout:max_xattr_size 65536 2024-05-03T18:47:21.849 INFO:teuthology.orchestra.run.smithi032.stdout:required_client_features {} 2024-05-03T18:47:21.849 INFO:teuthology.orchestra.run.smithi032.stdout:last_failure 0 2024-05-03T18:47:21.849 INFO:teuthology.orchestra.run.smithi032.stdout:last_failure_osd_epoch 0 2024-05-03T18:47:21.849 INFO:teuthology.orchestra.run.smithi032.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-03T18:47:21.849 INFO:teuthology.orchestra.run.smithi032.stdout:max_mds 3 2024-05-03T18:47:21.849 INFO:teuthology.orchestra.run.smithi032.stdout:in 0,1,2 2024-05-03T18:47:21.849 INFO:teuthology.orchestra.run.smithi032.stdout:up {0=24478,1=24469,2=24463} 2024-05-03T18:47:21.849 INFO:teuthology.orchestra.run.smithi032.stdout:failed 2024-05-03T18:47:21.849 INFO:teuthology.orchestra.run.smithi032.stdout:damaged 2024-05-03T18:47:21.850 INFO:teuthology.orchestra.run.smithi032.stdout:stopped 2024-05-03T18:47:21.850 INFO:teuthology.orchestra.run.smithi032.stdout:data_pools [3] 2024-05-03T18:47:21.850 INFO:teuthology.orchestra.run.smithi032.stdout:metadata_pool 2 2024-05-03T18:47:21.850 INFO:teuthology.orchestra.run.smithi032.stdout:inline_data disabled 2024-05-03T18:47:21.850 INFO:teuthology.orchestra.run.smithi032.stdout:balancer 2024-05-03T18:47:21.850 INFO:teuthology.orchestra.run.smithi032.stdout:bal_rank_mask -1 2024-05-03T18:47:21.850 INFO:teuthology.orchestra.run.smithi032.stdout:standby_count_wanted 0 2024-05-03T18:47:21.850 INFO:teuthology.orchestra.run.smithi032.stdout:qdb_cluster leader: 24478 members: 24469,24478,24463 2024-05-03T18:47:21.850 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.i{0:24478} state up:active seq 7 addr [v2:172.21.15.113:6838/942050693,v1:172.21.15.113:6839/942050693] compat {c=[1],r=[1],i=[1fff]}] 2024-05-03T18:47:21.850 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.b{0:24446} state up:standby-replay seq 1 addr [v2:172.21.15.45:6838/970804040,v1:172.21.15.45:6839/970804040] compat {c=[1],r=[1],i=[1fff]}] 2024-05-03T18:47:21.850 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.c{1:24469} state up:active seq 12 addr [v2:172.21.15.113:6836/3677284460,v1:172.21.15.113:6837/3677284460] compat {c=[1],r=[1],i=[1fff]}] 2024-05-03T18:47:21.850 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.l{1:24430} state up:standby-replay seq 1 addr [v2:172.21.15.113:6832/3323558791,v1:172.21.15.113:6833/3323558791] compat {c=[1],r=[1],i=[1fff]}] 2024-05-03T18:47:21.850 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.f{2:24463} state up:active seq 12 addr [v2:172.21.15.113:6834/2588738276,v1:172.21.15.113:6835/2588738276] compat {c=[1],r=[1],i=[1fff]}] 2024-05-03T18:47:21.850 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.h{2:24425} state up:standby-replay seq 1 addr [v2:172.21.15.45:6836/428055392,v1:172.21.15.45:6837/428055392] compat {c=[1],r=[1],i=[1fff]}] 2024-05-03T18:47:21.850 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-03T18:47:21.851 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-03T18:47:21.851 INFO:teuthology.orchestra.run.smithi032.stdout:Standby daemons: 2024-05-03T18:47:21.851 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-03T18:47:21.851 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.j{-1:14550} state up:standby seq 1 addr [v2:172.21.15.32:6834/535599929,v1:172.21.15.32:6835/535599929] compat {c=[1],r=[1],i=[1fff]}] 2024-05-03T18:47:21.851 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.d{-1:14562} state up:standby seq 1 addr [v2:172.21.15.32:6836/3101679868,v1:172.21.15.32:6837/3101679868] compat {c=[1],r=[1],i=[1fff]}] 2024-05-03T18:47:21.851 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.g{-1:14610} state up:standby seq 1 addr [v2:172.21.15.32:6838/3838653921,v1:172.21.15.32:6839/3838653921] compat {c=[1],r=[1],i=[1fff]}] 2024-05-03T18:47:21.851 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.a{-1:14622} state up:standby seq 1 addr [v2:172.21.15.32:6840/3019648436,v1:172.21.15.32:6841/3019648436] compat {c=[1],r=[1],i=[1fff]}] 2024-05-03T18:47:21.851 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.k{-1:24380} state up:standby seq 1 addr [v2:172.21.15.45:6832/1629869435,v1:172.21.15.45:6833/1629869435] compat {c=[1],r=[1],i=[1fff]}] 2024-05-03T18:47:21.851 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.e{-1:24398} state up:standby seq 1 addr [v2:172.21.15.45:6834/2997291166,v1:172.21.15.45:6835/2997291166] compat {c=[1],r=[1],i=[1fff]}] 2024-05-03T18:47:21.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:21.848+0000 7f79a4029700 1 -- 172.21.15.32:0/3063729054 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f79880b2910 msgr2=0x7f79880b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:21.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:21.848+0000 7f79a4029700 1 --2- 172.21.15.32:0/3063729054 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f79880b2910 0x7f79880b4dd0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f79980038c0 tx=0x7f7998000bc0 comp rx=0 tx=0).stop 2024-05-03T18:47:21.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:21.848+0000 7f79a4029700 1 -- 172.21.15.32:0/3063729054 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f799c104720 msgr2=0x7f799c199c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:21.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:21.848+0000 7f79a4029700 1 --2- 172.21.15.32:0/3063729054 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f799c104720 0x7f799c199c30 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f798c00c950 tx=0x7f798c00cd10 comp rx=0 tx=0).stop 2024-05-03T18:47:21.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:21.848+0000 7f79a4029700 1 -- 172.21.15.32:0/3063729054 shutdown_connections 2024-05-03T18:47:21.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:21.848+0000 7f79a4029700 1 --2- 172.21.15.32:0/3063729054 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f79880b2910 0x7f79880b4dd0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:21.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:21.848+0000 7f79a4029700 1 --2- 172.21.15.32:0/3063729054 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f799c1050c0 0x7f799c1a0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:21.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:21.848+0000 7f79a4029700 1 --2- 172.21.15.32:0/3063729054 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f799c103520 0x7f799c1996f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:21.852 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:21.848+0000 7f79a4029700 1 --2- 172.21.15.32:0/3063729054 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f799c104720 0x7f799c199c30 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:21.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:21.848+0000 7f79a4029700 1 -- 172.21.15.32:0/3063729054 >> 172.21.15.32:0/3063729054 conn(0x7f799c0feac0 msgr2=0x7f799c107af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:47:21.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:21.849+0000 7f79a4029700 1 -- 172.21.15.32:0/3063729054 shutdown_connections 2024-05-03T18:47:21.853 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:21.849+0000 7f79a4029700 1 -- 172.21.15.32:0/3063729054 wait complete. 2024-05-03T18:47:21.853 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 25 2024-05-03T18:47:22.501 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:47:22 smithi032 ceph-mon[29763]: from='client.? 172.21.15.32:0/3063729054' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-03T18:47:22.533 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f500455e-097b-11ef-bc97-c7b262605968 -- bash -c 'ceph osd dump' 2024-05-03T18:47:22.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:47:22 smithi113 ceph-mon[33540]: from='client.? 172.21.15.32:0/3063729054' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-03T18:47:22.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:47:22 smithi045 ceph-mon[33299]: from='client.? 172.21.15.32:0/3063729054' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-03T18:47:22.783 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:47:23.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:47:23 smithi113 ceph-mon[33540]: pgmap v360: 97 pgs: 97 active+clean; 631 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:47:23.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:47:23 smithi032 ceph-mon[29763]: pgmap v360: 97 pgs: 97 active+clean; 631 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:47:23.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:47:23 smithi045 ceph-mon[33299]: pgmap v360: 97 pgs: 97 active+clean; 631 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:47:24.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:24.340+0000 7f081ca6f700 1 -- 172.21.15.32:0/4042071567 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0818103f90 msgr2=0x7f081810a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:24.341 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:24.340+0000 7f081ca6f700 1 --2- 172.21.15.32:0/4042071567 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0818103f90 0x7f081810a840 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f081000b0d0 tx=0x7f081000b3e0 comp rx=0 tx=0).stop 2024-05-03T18:47:24.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:24.340+0000 7f081ca6f700 1 -- 172.21.15.32:0/4042071567 shutdown_connections 2024-05-03T18:47:24.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:24.340+0000 7f081ca6f700 1 --2- 172.21.15.32:0/4042071567 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0818103f90 0x7f081810a840 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:24.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:24.340+0000 7f081ca6f700 1 --2- 172.21.15.32:0/4042071567 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f08181023f0 0x7f0818102810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:24.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:24.340+0000 7f081ca6f700 1 --2- 172.21.15.32:0/4042071567 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08181035f0 0x7f0818103a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:24.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:24.340+0000 7f081ca6f700 1 -- 172.21.15.32:0/4042071567 >> 172.21.15.32:0/4042071567 conn(0x7f08180fd9d0 msgr2=0x7f08180ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:47:24.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:24.340+0000 7f081ca6f700 1 -- 172.21.15.32:0/4042071567 shutdown_connections 2024-05-03T18:47:24.342 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:24.340+0000 7f081ca6f700 1 -- 172.21.15.32:0/4042071567 wait complete. 2024-05-03T18:47:24.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:24.341+0000 7f081ca6f700 1 Processor -- start 2024-05-03T18:47:24.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:24.341+0000 7f081ca6f700 1 -- start start 2024-05-03T18:47:24.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:24.341+0000 7f081ca6f700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08181023f0 0x7f0818199590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:24.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:24.341+0000 7f081ca6f700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f08181035f0 0x7f0818199ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:24.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:24.341+0000 7f081ca6f700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0818103f90 0x7f08181a0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:24.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:24.341+0000 7f081ca6f700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f081810de80 con 0x7f0818103f90 2024-05-03T18:47:24.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:24.341+0000 7f081ca6f700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f081810dd00 con 0x7f08181023f0 2024-05-03T18:47:24.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:24.341+0000 7f081ca6f700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f081810e000 con 0x7f08181035f0 2024-05-03T18:47:24.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:24.341+0000 7f0816d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0818103f90 0x7f08181a0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:24.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:24.341+0000 7f0816d9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0818103f90 0x7f08181a0b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46506/0 (socket says 172.21.15.32:46506) 2024-05-03T18:47:24.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:24.341+0000 7f0816d9d700 1 -- 172.21.15.32:0/1350193160 learned_addr learned my addr 172.21.15.32:0/1350193160 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:47:24.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:24.341+0000 7f081659c700 1 --2- 172.21.15.32:0/1350193160 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08181023f0 0x7f0818199590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:24.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:24.341+0000 7f0815d9b700 1 --2- 172.21.15.32:0/1350193160 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f08181035f0 0x7f0818199ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:24.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:24.341+0000 7f0816d9d700 1 -- 172.21.15.32:0/1350193160 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f08181035f0 msgr2=0x7f0818199ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:24.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:24.341+0000 7f0816d9d700 1 --2- 172.21.15.32:0/1350193160 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f08181035f0 0x7f0818199ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:24.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:24.341+0000 7f0816d9d700 1 -- 172.21.15.32:0/1350193160 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08181023f0 msgr2=0x7f0818199590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:24.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:24.341+0000 7f0816d9d700 1 --2- 172.21.15.32:0/1350193160 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08181023f0 0x7f0818199590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:24.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:24.342+0000 7f0816d9d700 1 -- 172.21.15.32:0/1350193160 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08181a11f0 con 0x7f0818103f90 2024-05-03T18:47:24.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:24.342+0000 7f0815d9b700 1 --2- 172.21.15.32:0/1350193160 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f08181035f0 0x7f0818199ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:47:24.345 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:24.342+0000 7f0816d9d700 1 --2- 172.21.15.32:0/1350193160 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0818103f90 0x7f08181a0b50 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f0810009fd0 tx=0x7f0810004d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:47:24.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:24.342+0000 7f08077fe700 1 -- 172.21.15.32:0/1350193160 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f081001f030 con 0x7f0818103f90 2024-05-03T18:47:24.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:24.342+0000 7f08077fe700 1 -- 172.21.15.32:0/1350193160 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0810003a00 con 0x7f0818103f90 2024-05-03T18:47:24.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:24.342+0000 7f081ca6f700 1 -- 172.21.15.32:0/1350193160 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08181a1480 con 0x7f0818103f90 2024-05-03T18:47:24.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:24.342+0000 7f08077fe700 1 -- 172.21.15.32:0/1350193160 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08100046a0 con 0x7f0818103f90 2024-05-03T18:47:24.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:24.342+0000 7f081ca6f700 1 -- 172.21.15.32:0/1350193160 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08181a1930 con 0x7f0818103f90 2024-05-03T18:47:24.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:24.343+0000 7f081ca6f700 1 -- 172.21.15.32:0/1350193160 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f081806a190 con 0x7f0818103f90 2024-05-03T18:47:24.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:24.344+0000 7f08077fe700 1 -- 172.21.15.32:0/1350193160 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f0810003bb0 con 0x7f0818103f90 2024-05-03T18:47:24.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:24.345+0000 7f08077fe700 1 --2- 172.21.15.32:0/1350193160 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f08000b2990 0x7f08000b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:24.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:24.345+0000 7f081659c700 1 --2- 172.21.15.32:0/1350193160 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f08000b2990 0x7f08000b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:24.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:24.345+0000 7f08077fe700 1 -- 172.21.15.32:0/1350193160 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8356+0+0 (secure 0 0 0) 0x7f081000f020 con 0x7f0818103f90 2024-05-03T18:47:24.347 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:24.345+0000 7f081659c700 1 --2- 172.21.15.32:0/1350193160 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f08000b2990 0x7f08000b4e50 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f0808005950 tx=0x7f080800a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:47:24.349 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:24.348+0000 7f08077fe700 1 -- 172.21.15.32:0/1350193160 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f0810029020 con 0x7f0818103f90 2024-05-03T18:47:24.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:24.563+0000 7f081ca6f700 1 -- 172.21.15.32:0/1350193160 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f08181a1d50 con 0x7f0818103f90 2024-05-03T18:47:24.565 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:24.564+0000 7f08077fe700 1 -- 172.21.15.32:0/1350193160 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v82) ==== 56+0+5388 (secure 0 0 0) 0x7f08100ab670 con 0x7f0818103f90 2024-05-03T18:47:24.565 INFO:teuthology.orchestra.run.smithi032.stdout:epoch 82 2024-05-03T18:47:24.565 INFO:teuthology.orchestra.run.smithi032.stdout:fsid f500455e-097b-11ef-bc97-c7b262605968 2024-05-03T18:47:24.565 INFO:teuthology.orchestra.run.smithi032.stdout:created 2024-05-03T18:36:43.750772+0000 2024-05-03T18:47:24.565 INFO:teuthology.orchestra.run.smithi032.stdout:modified 2024-05-03T18:46:47.787519+0000 2024-05-03T18:47:24.565 INFO:teuthology.orchestra.run.smithi032.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-05-03T18:47:24.565 INFO:teuthology.orchestra.run.smithi032.stdout:crush_version 26 2024-05-03T18:47:24.566 INFO:teuthology.orchestra.run.smithi032.stdout:full_ratio 0.95 2024-05-03T18:47:24.566 INFO:teuthology.orchestra.run.smithi032.stdout:backfillfull_ratio 0.9 2024-05-03T18:47:24.566 INFO:teuthology.orchestra.run.smithi032.stdout:nearfull_ratio 0.85 2024-05-03T18:47:24.566 INFO:teuthology.orchestra.run.smithi032.stdout:require_min_compat_client luminous 2024-05-03T18:47:24.566 INFO:teuthology.orchestra.run.smithi032.stdout:min_compat_client luminous 2024-05-03T18:47:24.566 INFO:teuthology.orchestra.run.smithi032.stdout:require_osd_release squid 2024-05-03T18:47:24.566 INFO:teuthology.orchestra.run.smithi032.stdout:stretch_mode_enabled false 2024-05-03T18:47:24.566 INFO:teuthology.orchestra.run.smithi032.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-05-03T18:47:24.566 INFO:teuthology.orchestra.run.smithi032.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 78 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-05-03T18:47:24.566 INFO:teuthology.orchestra.run.smithi032.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 78 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-05-03T18:47:24.566 INFO:teuthology.orchestra.run.smithi032.stdout:max_osd 12 2024-05-03T18:47:24.566 INFO:teuthology.orchestra.run.smithi032.stdout:osd.0 up in weight 1 up_from 8 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.32:6802/262324956,v1:172.21.15.32:6803/262324956] [v2:172.21.15.32:6804/262324956,v1:172.21.15.32:6805/262324956] exists,up 0430c3c0-2c50-471c-9c74-2869a0627c74 2024-05-03T18:47:24.566 INFO:teuthology.orchestra.run.smithi032.stdout:osd.1 up in weight 1 up_from 13 up_thru 75 down_at 0 last_clean_interval [0,0) [v2:172.21.15.45:6800/1209907302,v1:172.21.15.45:6801/1209907302] [v2:172.21.15.45:6802/1209907302,v1:172.21.15.45:6803/1209907302] exists,up ba8f0b67-1d7d-4d4c-8276-eff664d21f02 2024-05-03T18:47:24.567 INFO:teuthology.orchestra.run.smithi032.stdout:osd.2 up in weight 1 up_from 18 up_thru 75 down_at 0 last_clean_interval [0,0) [v2:172.21.15.113:6800/2308128926,v1:172.21.15.113:6801/2308128926] [v2:172.21.15.113:6802/2308128926,v1:172.21.15.113:6803/2308128926] exists,up 5624eabc-be47-4bb8-9d8e-b84734769c83 2024-05-03T18:47:24.567 INFO:teuthology.orchestra.run.smithi032.stdout:osd.3 up in weight 1 up_from 25 up_thru 75 down_at 0 last_clean_interval [0,0) [v2:172.21.15.32:6810/2606514063,v1:172.21.15.32:6811/2606514063] [v2:172.21.15.32:6812/2606514063,v1:172.21.15.32:6813/2606514063] exists,up 62c9c977-afe1-4112-beba-b25af7caa2f7 2024-05-03T18:47:24.567 INFO:teuthology.orchestra.run.smithi032.stdout:osd.4 up in weight 1 up_from 31 up_thru 75 down_at 0 last_clean_interval [0,0) [v2:172.21.15.45:6808/71932118,v1:172.21.15.45:6809/71932118] [v2:172.21.15.45:6810/71932118,v1:172.21.15.45:6811/71932118] exists,up 99ad449f-b0bc-4aa1-b0cf-4a7a458126e6 2024-05-03T18:47:24.567 INFO:teuthology.orchestra.run.smithi032.stdout:osd.5 up in weight 1 up_from 36 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.113:6808/4085018597,v1:172.21.15.113:6809/4085018597] [v2:172.21.15.113:6810/4085018597,v1:172.21.15.113:6811/4085018597] exists,up a6688ade-47d7-4a52-807f-3c880a8ffcbb 2024-05-03T18:47:24.567 INFO:teuthology.orchestra.run.smithi032.stdout:osd.6 up in weight 1 up_from 42 up_thru 75 down_at 0 last_clean_interval [0,0) [v2:172.21.15.32:6818/791407796,v1:172.21.15.32:6819/791407796] [v2:172.21.15.32:6820/791407796,v1:172.21.15.32:6821/791407796] exists,up 1d9ebda9-55c7-473a-9dc0-fc60845bbfe9 2024-05-03T18:47:24.567 INFO:teuthology.orchestra.run.smithi032.stdout:osd.7 up in weight 1 up_from 47 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.45:6816/1032572438,v1:172.21.15.45:6817/1032572438] [v2:172.21.15.45:6818/1032572438,v1:172.21.15.45:6819/1032572438] exists,up 988cce65-23ed-4814-ac09-177eb2f89fe1 2024-05-03T18:47:24.567 INFO:teuthology.orchestra.run.smithi032.stdout:osd.8 up in weight 1 up_from 53 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.113:6816/3569181558,v1:172.21.15.113:6817/3569181558] [v2:172.21.15.113:6818/3569181558,v1:172.21.15.113:6819/3569181558] exists,up 816e8ce8-5afa-4b52-89be-f7793ed12a0b 2024-05-03T18:47:24.567 INFO:teuthology.orchestra.run.smithi032.stdout:osd.9 up in weight 1 up_from 59 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.32:6826/4290196345,v1:172.21.15.32:6827/4290196345] [v2:172.21.15.32:6828/4290196345,v1:172.21.15.32:6829/4290196345] exists,up 1d35c435-36b8-40e8-a2d4-0efe904f1107 2024-05-03T18:47:24.567 INFO:teuthology.orchestra.run.smithi032.stdout:osd.10 up in weight 1 up_from 64 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.45:6824/1817064361,v1:172.21.15.45:6825/1817064361] [v2:172.21.15.45:6826/1817064361,v1:172.21.15.45:6827/1817064361] exists,up 9af7879e-aa8d-4ef8-b289-4b145b614820 2024-05-03T18:47:24.567 INFO:teuthology.orchestra.run.smithi032.stdout:osd.11 up in weight 1 up_from 70 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.113:6824/703464306,v1:172.21.15.113:6825/703464306] [v2:172.21.15.113:6826/703464306,v1:172.21.15.113:6827/703464306] exists,up 22981709-1f9d-46f6-892a-5678394a2e1f 2024-05-03T18:47:24.567 INFO:teuthology.orchestra.run.smithi032.stdout:pg_upmap_items 2.5 [8,11] 2024-05-03T18:47:24.567 INFO:teuthology.orchestra.run.smithi032.stdout:pg_upmap_items 2.b [2,4] 2024-05-03T18:47:24.567 INFO:teuthology.orchestra.run.smithi032.stdout:pg_upmap_items 2.1d [2,1] 2024-05-03T18:47:24.567 INFO:teuthology.orchestra.run.smithi032.stdout:pg_upmap_items 2.1f [7,11] 2024-05-03T18:47:24.568 INFO:teuthology.orchestra.run.smithi032.stdout:pg_upmap_items 3.2 [3,9] 2024-05-03T18:47:24.568 INFO:teuthology.orchestra.run.smithi032.stdout:pg_upmap_items 3.c [5,8] 2024-05-03T18:47:24.568 INFO:teuthology.orchestra.run.smithi032.stdout:pg_upmap_items 3.e [4,7] 2024-05-03T18:47:24.568 INFO:teuthology.orchestra.run.smithi032.stdout:pg_upmap_items 3.17 [4,6] 2024-05-03T18:47:24.568 INFO:teuthology.orchestra.run.smithi032.stdout:pg_upmap_items 3.1b [4,6] 2024-05-03T18:47:24.568 INFO:teuthology.orchestra.run.smithi032.stdout:pg_upmap_items 3.2f [1,9] 2024-05-03T18:47:24.568 INFO:teuthology.orchestra.run.smithi032.stdout:pg_upmap_items 3.39 [5,8] 2024-05-03T18:47:24.568 INFO:teuthology.orchestra.run.smithi032.stdout:pg_upmap_items 3.3a [4,8] 2024-05-03T18:47:24.568 INFO:teuthology.orchestra.run.smithi032.stdout:pg_upmap_items 3.3b [3,9] 2024-05-03T18:47:24.568 INFO:teuthology.orchestra.run.smithi032.stdout:blocklist 172.21.15.32:0/743758274 expires 2024-05-04T18:37:45.879602+0000 2024-05-03T18:47:24.568 INFO:teuthology.orchestra.run.smithi032.stdout:blocklist 172.21.15.32:6801/2019702098 expires 2024-05-04T18:37:45.879602+0000 2024-05-03T18:47:24.568 INFO:teuthology.orchestra.run.smithi032.stdout:blocklist 172.21.15.32:6800/2019702098 expires 2024-05-04T18:37:45.879602+0000 2024-05-03T18:47:24.568 INFO:teuthology.orchestra.run.smithi032.stdout:blocklist 172.21.15.32:0/3192385439 expires 2024-05-04T18:37:18.492319+0000 2024-05-03T18:47:24.569 INFO:teuthology.orchestra.run.smithi032.stdout:blocklist 172.21.15.32:0/1450308494 expires 2024-05-04T18:37:18.492319+0000 2024-05-03T18:47:24.569 INFO:teuthology.orchestra.run.smithi032.stdout:blocklist 172.21.15.32:0/3957699240 expires 2024-05-04T18:37:45.879602+0000 2024-05-03T18:47:24.569 INFO:teuthology.orchestra.run.smithi032.stdout:blocklist 172.21.15.32:0/3310503393 expires 2024-05-04T18:37:18.492319+0000 2024-05-03T18:47:24.569 INFO:teuthology.orchestra.run.smithi032.stdout:blocklist 172.21.15.32:6801/3945405684 expires 2024-05-04T18:37:18.492319+0000 2024-05-03T18:47:24.569 INFO:teuthology.orchestra.run.smithi032.stdout:blocklist 172.21.15.32:6800/3945405684 expires 2024-05-04T18:37:18.492319+0000 2024-05-03T18:47:24.569 INFO:teuthology.orchestra.run.smithi032.stdout:blocklist 172.21.15.32:0/2471164468 expires 2024-05-04T18:37:45.879602+0000 2024-05-03T18:47:24.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:24.566+0000 7f081ca6f700 1 -- 172.21.15.32:0/1350193160 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f08000b2990 msgr2=0x7f08000b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:24.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:24.566+0000 7f081ca6f700 1 --2- 172.21.15.32:0/1350193160 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f08000b2990 0x7f08000b4e50 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f0808005950 tx=0x7f080800a300 comp rx=0 tx=0).stop 2024-05-03T18:47:24.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:24.566+0000 7f081ca6f700 1 -- 172.21.15.32:0/1350193160 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0818103f90 msgr2=0x7f08181a0b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:24.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:24.566+0000 7f081ca6f700 1 --2- 172.21.15.32:0/1350193160 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0818103f90 0x7f08181a0b50 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f0810009fd0 tx=0x7f0810004d50 comp rx=0 tx=0).stop 2024-05-03T18:47:24.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:24.566+0000 7f081ca6f700 1 -- 172.21.15.32:0/1350193160 shutdown_connections 2024-05-03T18:47:24.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:24.566+0000 7f081ca6f700 1 --2- 172.21.15.32:0/1350193160 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f08000b2990 0x7f08000b4e50 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:24.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:24.566+0000 7f081ca6f700 1 --2- 172.21.15.32:0/1350193160 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0818103f90 0x7f08181a0b50 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:24.570 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:24.566+0000 7f081ca6f700 1 --2- 172.21.15.32:0/1350193160 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08181023f0 0x7f0818199590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:24.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:24.566+0000 7f081ca6f700 1 --2- 172.21.15.32:0/1350193160 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f08181035f0 0x7f0818199ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:24.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:24.566+0000 7f081ca6f700 1 -- 172.21.15.32:0/1350193160 >> 172.21.15.32:0/1350193160 conn(0x7f08180fd9d0 msgr2=0x7f08180ff460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:47:24.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:24.566+0000 7f081ca6f700 1 -- 172.21.15.32:0/1350193160 shutdown_connections 2024-05-03T18:47:24.571 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:24.567+0000 7f081ca6f700 1 -- 172.21.15.32:0/1350193160 wait complete. 2024-05-03T18:47:25.255 INFO:teuthology.run_tasks:Running task fs.ready... 2024-05-03T18:47:25.267 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid f500455e-097b-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-03T18:47:25.520 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:47:25.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:47:25 smithi113 ceph-mon[33540]: pgmap v361: 97 pgs: 97 active+clean; 631 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:47:25.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:47:25 smithi113 ceph-mon[33540]: from='client.? 172.21.15.32:0/1350193160' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-03T18:47:25.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:47:25 smithi032 ceph-mon[29763]: pgmap v361: 97 pgs: 97 active+clean; 631 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:47:25.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:47:25 smithi032 ceph-mon[29763]: from='client.? 172.21.15.32:0/1350193160' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-03T18:47:25.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:47:25 smithi045 ceph-mon[33299]: pgmap v361: 97 pgs: 97 active+clean; 631 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:47:25.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:47:25 smithi045 ceph-mon[33299]: from='client.? 172.21.15.32:0/1350193160' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-03T18:47:27.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:27.230+0000 7fb8bd191700 1 -- 172.21.15.32:0/2111963046 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8b8108410 msgr2=0x7fb8b81087f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:27.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:27.230+0000 7fb8bd191700 1 --2- 172.21.15.32:0/2111963046 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8b8108410 0x7fb8b81087f0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fb8a8009990 tx=0x7fb8a8009ca0 comp rx=0 tx=0).stop 2024-05-03T18:47:27.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:27.230+0000 7fb8bd191700 1 -- 172.21.15.32:0/2111963046 shutdown_connections 2024-05-03T18:47:27.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:27.230+0000 7fb8bd191700 1 --2- 172.21.15.32:0/2111963046 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb8b8102d60 0x7fb8b810f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:27.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:27.230+0000 7fb8bd191700 1 --2- 172.21.15.32:0/2111963046 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8b8108410 0x7fb8b81087f0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:27.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:27.230+0000 7fb8bd191700 1 --2- 172.21.15.32:0/2111963046 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8b81023a0 0x7fb8b8102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:27.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:27.230+0000 7fb8bd191700 1 -- 172.21.15.32:0/2111963046 >> 172.21.15.32:0/2111963046 conn(0x7fb8b8079350 msgr2=0x7fb8b8079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:47:27.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:27.230+0000 7fb8bd191700 1 -- 172.21.15.32:0/2111963046 shutdown_connections 2024-05-03T18:47:27.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:27.230+0000 7fb8bd191700 1 -- 172.21.15.32:0/2111963046 wait complete. 2024-05-03T18:47:27.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:27.231+0000 7fb8bd191700 1 Processor -- start 2024-05-03T18:47:27.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:27.231+0000 7fb8bd191700 1 -- start start 2024-05-03T18:47:27.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:27.231+0000 7fb8bd191700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb8b81023a0 0x7fb8b819f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:27.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:27.231+0000 7fb8bd191700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8b8102d60 0x7fb8b819fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:27.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:27.231+0000 7fb8bd191700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8b8108410 0x7fb8b81999d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:27.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:27.231+0000 7fb8bd191700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fb8b8112560 con 0x7fb8b8108410 2024-05-03T18:47:27.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:27.231+0000 7fb8bd191700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb8b81123e0 con 0x7fb8b8102d60 2024-05-03T18:47:27.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:27.231+0000 7fb8bd191700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fb8b81126e0 con 0x7fb8b81023a0 2024-05-03T18:47:27.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:27.232+0000 7fb8b759e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8b8108410 0x7fb8b81999d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:27.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:27.232+0000 7fb8b6d9d700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb8b81023a0 0x7fb8b819f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:27.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:27.232+0000 7fb8b6d9d700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb8b81023a0 0x7fb8b819f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.32:37504/0 (socket says 172.21.15.32:37504) 2024-05-03T18:47:27.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:27.232+0000 7fb8b759e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8b8108410 0x7fb8b81999d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46520/0 (socket says 172.21.15.32:46520) 2024-05-03T18:47:27.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:27.232+0000 7fb8b6d9d700 1 -- 172.21.15.32:0/724575243 learned_addr learned my addr 172.21.15.32:0/724575243 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:47:27.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:27.232+0000 7fb8b659c700 1 --2- 172.21.15.32:0/724575243 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8b8102d60 0x7fb8b819fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:27.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:27.232+0000 7fb8b759e700 1 -- 172.21.15.32:0/724575243 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb8b81023a0 msgr2=0x7fb8b819f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:27.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:27.232+0000 7fb8b759e700 1 --2- 172.21.15.32:0/724575243 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb8b81023a0 0x7fb8b819f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:27.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:27.232+0000 7fb8b759e700 1 -- 172.21.15.32:0/724575243 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8b8102d60 msgr2=0x7fb8b819fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:27.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:27.232+0000 7fb8b759e700 1 --2- 172.21.15.32:0/724575243 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8b8102d60 0x7fb8b819fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:27.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:27.232+0000 7fb8b759e700 1 -- 172.21.15.32:0/724575243 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8b819a130 con 0x7fb8b8108410 2024-05-03T18:47:27.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:27.232+0000 7fb8b6d9d700 1 --2- 172.21.15.32:0/724575243 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb8b81023a0 0x7fb8b819f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-03T18:47:27.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:27.232+0000 7fb8b659c700 1 --2- 172.21.15.32:0/724575243 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8b8102d60 0x7fb8b819fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T18:47:27.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:27.232+0000 7fb8b759e700 1 --2- 172.21.15.32:0/724575243 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8b8108410 0x7fb8b81999d0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fb8b0007f10 tx=0x7fb8b000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:47:27.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:27.232+0000 7fb89ffff700 1 -- 172.21.15.32:0/724575243 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8b000ddd0 con 0x7fb8b8108410 2024-05-03T18:47:27.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:27.232+0000 7fb89ffff700 1 -- 172.21.15.32:0/724575243 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb8b0009e20 con 0x7fb8b8108410 2024-05-03T18:47:27.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:27.232+0000 7fb8bd191700 1 -- 172.21.15.32:0/724575243 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8b819a420 con 0x7fb8b8108410 2024-05-03T18:47:27.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:27.232+0000 7fb89ffff700 1 -- 172.21.15.32:0/724575243 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8b0015400 con 0x7fb8b8108410 2024-05-03T18:47:27.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:27.232+0000 7fb8bd191700 1 -- 172.21.15.32:0/724575243 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8b81a66e0 con 0x7fb8b8108410 2024-05-03T18:47:27.237 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:27.233+0000 7fb8bd191700 1 -- 172.21.15.32:0/724575243 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8b806a190 con 0x7fb8b8108410 2024-05-03T18:47:27.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:27.235+0000 7fb89ffff700 1 -- 172.21.15.32:0/724575243 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fb8b0005000 con 0x7fb8b8108410 2024-05-03T18:47:27.238 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:27.236+0000 7fb89ffff700 1 --2- 172.21.15.32:0/724575243 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fb8a00b2910 0x7fb8a00b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:27.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:27.236+0000 7fb8b6d9d700 1 --2- 172.21.15.32:0/724575243 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fb8a00b2910 0x7fb8a00b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:27.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:27.236+0000 7fb89ffff700 1 -- 172.21.15.32:0/724575243 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8356+0+0 (secure 0 0 0) 0x7fb8b00e25e0 con 0x7fb8b8108410 2024-05-03T18:47:27.239 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:27.236+0000 7fb8b6d9d700 1 --2- 172.21.15.32:0/724575243 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fb8a00b2910 0x7fb8a00b4dd0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fb8a8003800 tx=0x7fb8a8005e80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:47:27.240 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:27.239+0000 7fb89ffff700 1 -- 172.21.15.32:0/724575243 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fb8b0010070 con 0x7fb8b8108410 2024-05-03T18:47:27.295 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:47:27 smithi032 ceph-mon[29763]: pgmap v362: 97 pgs: 97 active+clean; 631 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:47:27.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:27.502+0000 7fb8bd191700 1 -- 172.21.15.32:0/724575243 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb8b81a68b0 con 0x7fb8b8108410 2024-05-03T18:47:27.503 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:27.502+0000 7fb89ffff700 1 -- 172.21.15.32:0/724575243 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11862 (secure 0 0 0) 0x7fb8b00aa120 con 0x7fb8b8108410 2024-05-03T18:47:27.503 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-03T18:47:27.503 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/535599929","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":535599929},{"type":"v1","addr":"172.21.15.32:6835","nonce":535599929}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/3101679868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":3101679868},{"type":"v1","addr":"172.21.15.32:6837","nonce":3101679868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14610,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/3838653921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":3838653921},{"type":"v1","addr":"172.21.15.32:6839","nonce":3838653921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14622,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6841/3019648436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6840","nonce":3019648436},{"type":"v1","addr":"172.21.15.32:6841","nonce":3019648436}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24380,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/1629869435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":1629869435},{"type":"v1","addr":"172.21.15.45:6833","nonce":1629869435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24398,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2997291166","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2997291166},{"type":"v1","addr":"172.21.15.45:6835","nonce":2997291166}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-03T18:44:51.155050+0000","modified":"2024-05-03T18:45:08.404530+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24478,"mds_1":24469,"mds_2":24463},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24425":{"gid":24425,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.45:6837/428055392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":428055392},{"type":"v1","addr":"172.21.15.45:6837","nonce":428055392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.113:6833/3323558791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":3323558791},{"type":"v1","addr":"172.21.15.113:6833","nonce":3323558791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"b","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.45:6839/970804040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":970804040},{"type":"v1","addr":"172.21.15.45:6839","nonce":970804040}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.113:6835/2588738276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2588738276},{"type":"v1","addr":"172.21.15.113:6835","nonce":2588738276}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.113:6837/3677284460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":3677284460},{"type":"v1","addr":"172.21.15.113:6837","nonce":3677284460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.113:6839/942050693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6838","nonce":942050693},{"type":"v1","addr":"172.21.15.113:6839","nonce":942050693}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24478,"qdb_cluster":[24469,24478,24463]},"id":1}]} 2024-05-03T18:47:27.506 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:27.504+0000 7fb8bd191700 1 -- 172.21.15.32:0/724575243 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fb8a00b2910 msgr2=0x7fb8a00b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:27.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:27.504+0000 7fb8bd191700 1 --2- 172.21.15.32:0/724575243 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fb8a00b2910 0x7fb8a00b4dd0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fb8a8003800 tx=0x7fb8a8005e80 comp rx=0 tx=0).stop 2024-05-03T18:47:27.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:27.504+0000 7fb8bd191700 1 -- 172.21.15.32:0/724575243 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8b8108410 msgr2=0x7fb8b81999d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:27.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:27.504+0000 7fb8bd191700 1 --2- 172.21.15.32:0/724575243 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8b8108410 0x7fb8b81999d0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fb8b0007f10 tx=0x7fb8b000d3b0 comp rx=0 tx=0).stop 2024-05-03T18:47:27.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:27.505+0000 7fb8bd191700 1 -- 172.21.15.32:0/724575243 shutdown_connections 2024-05-03T18:47:27.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:27.505+0000 7fb8bd191700 1 --2- 172.21.15.32:0/724575243 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fb8a00b2910 0x7fb8a00b4dd0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:27.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:27.505+0000 7fb8bd191700 1 --2- 172.21.15.32:0/724575243 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8b8108410 0x7fb8b81999d0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:27.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:27.505+0000 7fb8bd191700 1 --2- 172.21.15.32:0/724575243 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb8b81023a0 0x7fb8b819f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:27.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:27.505+0000 7fb8bd191700 1 --2- 172.21.15.32:0/724575243 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8b8102d60 0x7fb8b819fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:27.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:27.505+0000 7fb8bd191700 1 -- 172.21.15.32:0/724575243 >> 172.21.15.32:0/724575243 conn(0x7fb8b8079350 msgr2=0x7fb8b81017b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:47:27.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:27.505+0000 7fb8bd191700 1 -- 172.21.15.32:0/724575243 shutdown_connections 2024-05-03T18:47:27.507 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:27.505+0000 7fb8bd191700 1 -- 172.21.15.32:0/724575243 wait complete. 2024-05-03T18:47:27.507 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 25 2024-05-03T18:47:27.550 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:47:27 smithi045 ceph-mon[33299]: pgmap v362: 97 pgs: 97 active+clean; 631 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:47:27.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:47:27 smithi113 ceph-mon[33540]: pgmap v362: 97 pgs: 97 active+clean; 631 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:47:28.120 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid f500455e-097b-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-03T18:47:28.370 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:47:28.490 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:47:28 smithi032 ceph-mon[29763]: from='client.? 172.21.15.32:0/724575243' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:47:28.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:47:28 smithi045 ceph-mon[33299]: from='client.? 172.21.15.32:0/724575243' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:47:28.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:47:28 smithi113 ceph-mon[33540]: from='client.? 172.21.15.32:0/724575243' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:47:29.471 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:47:29 smithi032 ceph-mon[29763]: pgmap v363: 97 pgs: 97 active+clean; 631 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:47:29.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:47:29 smithi045 ceph-mon[33299]: pgmap v363: 97 pgs: 97 active+clean; 631 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:47:29.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:47:29 smithi113 ceph-mon[33540]: pgmap v363: 97 pgs: 97 active+clean; 631 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:47:29.996 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:29.995+0000 7f14bd943700 1 -- 172.21.15.32:0/4220019820 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14b8102fe0 msgr2=0x7f14b8106fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:29.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:29.995+0000 7f14bd943700 1 --2- 172.21.15.32:0/4220019820 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14b8102fe0 0x7f14b8106fe0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f14ac0099c0 tx=0x7f14ac009cd0 comp rx=0 tx=0).stop 2024-05-03T18:47:29.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:29.995+0000 7f14bd943700 1 -- 172.21.15.32:0/4220019820 shutdown_connections 2024-05-03T18:47:29.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:29.995+0000 7f14bd943700 1 --2- 172.21.15.32:0/4220019820 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f14b81076c0 0x7f14b810f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:29.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:29.995+0000 7f14bd943700 1 --2- 172.21.15.32:0/4220019820 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14b8102630 0x7f14b8102a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:29.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:29.995+0000 7f14bd943700 1 --2- 172.21.15.32:0/4220019820 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14b8102fe0 0x7f14b8106fe0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:29.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:29.995+0000 7f14bd943700 1 -- 172.21.15.32:0/4220019820 >> 172.21.15.32:0/4220019820 conn(0x7f14b8079350 msgr2=0x7f14b8079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:47:29.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:29.995+0000 7f14bd943700 1 -- 172.21.15.32:0/4220019820 shutdown_connections 2024-05-03T18:47:29.997 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:29.995+0000 7f14bd943700 1 -- 172.21.15.32:0/4220019820 wait complete. 2024-05-03T18:47:29.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:29.996+0000 7f14bd943700 1 Processor -- start 2024-05-03T18:47:29.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:29.996+0000 7f14bd943700 1 -- start start 2024-05-03T18:47:29.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:29.997+0000 7f14bd943700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14b8102630 0x7f14b819f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:29.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:29.997+0000 7f14bd943700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f14b8102fe0 0x7f14b819fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:29.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:29.997+0000 7f14bd943700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14b81076c0 0x7f14b8199b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:29.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:29.997+0000 7f14bd943700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f14b8112630 con 0x7f14b8102630 2024-05-03T18:47:29.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:29.997+0000 7f14b6ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14b8102630 0x7f14b819f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:29.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:29.997+0000 7f14bd943700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f14b81124b0 con 0x7f14b81076c0 2024-05-03T18:47:29.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:29.997+0000 7f14b6ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14b8102630 0x7f14b819f940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46538/0 (socket says 172.21.15.32:46538) 2024-05-03T18:47:29.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:29.997+0000 7f14b6ffd700 1 -- 172.21.15.32:0/1536918722 learned_addr learned my addr 172.21.15.32:0/1536918722 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:47:29.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:29.997+0000 7f14bd943700 1 -- 172.21.15.32:0/1536918722 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f14b81127b0 con 0x7f14b8102fe0 2024-05-03T18:47:29.999 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:29.997+0000 7f14b77fe700 1 --2- 172.21.15.32:0/1536918722 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14b81076c0 0x7f14b8199b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:30.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:29.997+0000 7f14b67fc700 1 --2- 172.21.15.32:0/1536918722 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f14b8102fe0 0x7f14b819fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:30.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:29.997+0000 7f14b6ffd700 1 -- 172.21.15.32:0/1536918722 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f14b8102fe0 msgr2=0x7f14b819fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:30.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:29.997+0000 7f14b6ffd700 1 --2- 172.21.15.32:0/1536918722 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f14b8102fe0 0x7f14b819fe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:30.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:29.997+0000 7f14b6ffd700 1 -- 172.21.15.32:0/1536918722 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14b81076c0 msgr2=0x7f14b8199b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:30.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:29.997+0000 7f14b6ffd700 1 --2- 172.21.15.32:0/1536918722 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14b81076c0 0x7f14b8199b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:30.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:29.997+0000 7f14b6ffd700 1 -- 172.21.15.32:0/1536918722 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14b819a260 con 0x7f14b8102630 2024-05-03T18:47:30.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:29.997+0000 7f14b6ffd700 1 --2- 172.21.15.32:0/1536918722 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14b8102630 0x7f14b819f940 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f14a800b530 tx=0x7f14a800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:47:30.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:29.998+0000 7f14bc941700 1 -- 172.21.15.32:0/1536918722 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14a8015020 con 0x7f14b8102630 2024-05-03T18:47:30.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:29.998+0000 7f14bd943700 1 -- 172.21.15.32:0/1536918722 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14b819a550 con 0x7f14b8102630 2024-05-03T18:47:30.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:29.998+0000 7f14bc941700 1 -- 172.21.15.32:0/1536918722 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14a8011ce0 con 0x7f14b8102630 2024-05-03T18:47:30.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:29.998+0000 7f14bc941700 1 -- 172.21.15.32:0/1536918722 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14a8010dd0 con 0x7f14b8102630 2024-05-03T18:47:30.000 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:29.998+0000 7f14bd943700 1 -- 172.21.15.32:0/1536918722 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14b81a6810 con 0x7f14b8102630 2024-05-03T18:47:30.001 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:29.999+0000 7f14bd943700 1 -- 172.21.15.32:0/1536918722 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14b8103ac0 con 0x7f14b8102630 2024-05-03T18:47:30.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:30.001+0000 7f14bc941700 1 -- 172.21.15.32:0/1536918722 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f14a8018020 con 0x7f14b8102630 2024-05-03T18:47:30.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:30.001+0000 7f14bc941700 1 --2- 172.21.15.32:0/1536918722 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f14a00b2910 0x7f14a00b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:30.002 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:30.002+0000 7f14b67fc700 1 --2- 172.21.15.32:0/1536918722 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f14a00b2910 0x7f14a00b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:30.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:30.002+0000 7f14bc941700 1 -- 172.21.15.32:0/1536918722 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8356+0+0 (secure 0 0 0) 0x7f14a80e1fd0 con 0x7f14b8102630 2024-05-03T18:47:30.003 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:30.002+0000 7f14b67fc700 1 --2- 172.21.15.32:0/1536918722 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f14a00b2910 0x7f14a00b4dd0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f14ac006010 tx=0x7f14ac0096b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:47:30.006 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:30.005+0000 7f14bc941700 1 -- 172.21.15.32:0/1536918722 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f14a801f020 con 0x7f14b8102630 2024-05-03T18:47:30.271 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:30.270+0000 7f14bd943700 1 -- 172.21.15.32:0/1536918722 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f14b819aff0 con 0x7f14b8102630 2024-05-03T18:47:30.272 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:30.271+0000 7f14bc941700 1 -- 172.21.15.32:0/1536918722 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11862 (secure 0 0 0) 0x7f14a80a9bc0 con 0x7f14b8102630 2024-05-03T18:47:30.272 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-03T18:47:30.272 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/535599929","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":535599929},{"type":"v1","addr":"172.21.15.32:6835","nonce":535599929}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/3101679868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":3101679868},{"type":"v1","addr":"172.21.15.32:6837","nonce":3101679868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14610,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/3838653921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":3838653921},{"type":"v1","addr":"172.21.15.32:6839","nonce":3838653921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14622,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6841/3019648436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6840","nonce":3019648436},{"type":"v1","addr":"172.21.15.32:6841","nonce":3019648436}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24380,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/1629869435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":1629869435},{"type":"v1","addr":"172.21.15.45:6833","nonce":1629869435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24398,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2997291166","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2997291166},{"type":"v1","addr":"172.21.15.45:6835","nonce":2997291166}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-03T18:44:51.155050+0000","modified":"2024-05-03T18:45:08.404530+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24478,"mds_1":24469,"mds_2":24463},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24425":{"gid":24425,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.45:6837/428055392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":428055392},{"type":"v1","addr":"172.21.15.45:6837","nonce":428055392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.113:6833/3323558791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":3323558791},{"type":"v1","addr":"172.21.15.113:6833","nonce":3323558791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"b","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.45:6839/970804040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":970804040},{"type":"v1","addr":"172.21.15.45:6839","nonce":970804040}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.113:6835/2588738276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2588738276},{"type":"v1","addr":"172.21.15.113:6835","nonce":2588738276}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.113:6837/3677284460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":3677284460},{"type":"v1","addr":"172.21.15.113:6837","nonce":3677284460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.113:6839/942050693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6838","nonce":942050693},{"type":"v1","addr":"172.21.15.113:6839","nonce":942050693}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24478,"qdb_cluster":[24469,24478,24463]},"id":1}]} 2024-05-03T18:47:30.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:30.273+0000 7f14bd943700 1 -- 172.21.15.32:0/1536918722 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f14a00b2910 msgr2=0x7f14a00b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:30.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:30.273+0000 7f14bd943700 1 --2- 172.21.15.32:0/1536918722 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f14a00b2910 0x7f14a00b4dd0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f14ac006010 tx=0x7f14ac0096b0 comp rx=0 tx=0).stop 2024-05-03T18:47:30.274 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:30.273+0000 7f14bd943700 1 -- 172.21.15.32:0/1536918722 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14b8102630 msgr2=0x7f14b819f940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:30.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:30.273+0000 7f14bd943700 1 --2- 172.21.15.32:0/1536918722 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14b8102630 0x7f14b819f940 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f14a800b530 tx=0x7f14a800b840 comp rx=0 tx=0).stop 2024-05-03T18:47:30.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:30.273+0000 7f14bd943700 1 -- 172.21.15.32:0/1536918722 shutdown_connections 2024-05-03T18:47:30.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:30.273+0000 7f14bd943700 1 --2- 172.21.15.32:0/1536918722 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f14a00b2910 0x7f14a00b4dd0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:30.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:30.273+0000 7f14bd943700 1 --2- 172.21.15.32:0/1536918722 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14b81076c0 0x7f14b8199b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:30.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:30.273+0000 7f14bd943700 1 --2- 172.21.15.32:0/1536918722 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14b8102630 0x7f14b819f940 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:30.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:30.273+0000 7f14bd943700 1 --2- 172.21.15.32:0/1536918722 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f14b8102fe0 0x7f14b819fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:30.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:30.273+0000 7f14bd943700 1 -- 172.21.15.32:0/1536918722 >> 172.21.15.32:0/1536918722 conn(0x7f14b8079350 msgr2=0x7f14b806bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:47:30.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:30.273+0000 7f14bd943700 1 -- 172.21.15.32:0/1536918722 shutdown_connections 2024-05-03T18:47:30.275 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:30.273+0000 7f14bd943700 1 -- 172.21.15.32:0/1536918722 wait complete. 2024-05-03T18:47:30.275 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 25 2024-05-03T18:47:30.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:47:30 smithi032 ceph-mon[29763]: from='client.? 172.21.15.32:0/1536918722' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:47:30.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:47:30 smithi045 ceph-mon[33299]: from='client.? 172.21.15.32:0/1536918722' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:47:30.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:47:30 smithi113 ceph-mon[33540]: from='client.? 172.21.15.32:0/1536918722' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T18:47:30.925 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid f500455e-097b-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-03T18:47:31.182 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:47:31.564 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:47:31 smithi032 ceph-mon[29763]: pgmap v364: 97 pgs: 97 active+clean; 631 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:47:31.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:47:31 smithi045 ceph-mon[33299]: pgmap v364: 97 pgs: 97 active+clean; 631 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:47:31.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:47:31 smithi113 ceph-mon[33540]: pgmap v364: 97 pgs: 97 active+clean; 631 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:47:32.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:32.786+0000 7f086995a700 1 -- 172.21.15.32:0/2737306338 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0864103f00 msgr2=0x7f08641103b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:32.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:32.786+0000 7f086995a700 1 --2- 172.21.15.32:0/2737306338 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0864103f00 0x7f08641103b0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f08540099c0 tx=0x7f0854009cd0 comp rx=0 tx=0).stop 2024-05-03T18:47:32.789 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:32.786+0000 7f086995a700 1 -- 172.21.15.32:0/2737306338 shutdown_connections 2024-05-03T18:47:32.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:32.786+0000 7f086995a700 1 --2- 172.21.15.32:0/2737306338 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0864103f00 0x7f08641103b0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:32.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:32.786+0000 7f086995a700 1 --2- 172.21.15.32:0/2737306338 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08641095e0 0x7f08641099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:32.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:32.786+0000 7f086995a700 1 --2- 172.21.15.32:0/2737306338 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0864103540 0x7f08641039c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:32.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:32.786+0000 7f086995a700 1 -- 172.21.15.32:0/2737306338 >> 172.21.15.32:0/2737306338 conn(0x7f086407a650 msgr2=0x7f086407aa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:47:32.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:32.786+0000 7f086995a700 1 -- 172.21.15.32:0/2737306338 shutdown_connections 2024-05-03T18:47:32.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:32.786+0000 7f086995a700 1 -- 172.21.15.32:0/2737306338 wait complete. 2024-05-03T18:47:32.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:32.787+0000 7f086995a700 1 Processor -- start 2024-05-03T18:47:32.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:32.787+0000 7f086995a700 1 -- start start 2024-05-03T18:47:32.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:32.787+0000 7f086995a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0864103540 0x7f086419f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:32.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:32.787+0000 7f086995a700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0864103f00 0x7f086419fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:32.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:32.787+0000 7f086995a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08641095e0 0x7f0864199a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:32.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:32.787+0000 7f086995a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f08641137c0 con 0x7f0864103540 2024-05-03T18:47:32.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:32.787+0000 7f086995a700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f0864113640 con 0x7f08641095e0 2024-05-03T18:47:32.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:32.787+0000 7f086995a700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f0864113940 con 0x7f0864103f00 2024-05-03T18:47:32.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:32.787+0000 7f0862ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0864103540 0x7f086419f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:32.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:32.787+0000 7f0862ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0864103540 0x7f086419f860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46560/0 (socket says 172.21.15.32:46560) 2024-05-03T18:47:32.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:32.787+0000 7f0862ffd700 1 -- 172.21.15.32:0/438474416 learned_addr learned my addr 172.21.15.32:0/438474416 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:47:32.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:32.787+0000 7f08627fc700 1 --2- 172.21.15.32:0/438474416 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0864103f00 0x7f086419fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:32.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:32.787+0000 7f08637fe700 1 --2- 172.21.15.32:0/438474416 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08641095e0 0x7f0864199a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:32.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:32.787+0000 7f0862ffd700 1 -- 172.21.15.32:0/438474416 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0864103f00 msgr2=0x7f086419fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:32.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:32.788+0000 7f0862ffd700 1 --2- 172.21.15.32:0/438474416 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0864103f00 0x7f086419fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:32.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:32.788+0000 7f0862ffd700 1 -- 172.21.15.32:0/438474416 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08641095e0 msgr2=0x7f0864199a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:32.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:32.788+0000 7f0862ffd700 1 --2- 172.21.15.32:0/438474416 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08641095e0 0x7f0864199a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:32.791 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:32.788+0000 7f0862ffd700 1 -- 172.21.15.32:0/438474416 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f086419a180 con 0x7f0864103540 2024-05-03T18:47:32.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:32.788+0000 7f0862ffd700 1 --2- 172.21.15.32:0/438474416 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0864103540 0x7f086419f860 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f085c00ba50 tx=0x7f085c00bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:47:32.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:32.788+0000 7f0868958700 1 -- 172.21.15.32:0/438474416 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f085c019070 con 0x7f0864103540 2024-05-03T18:47:32.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:32.788+0000 7f0868958700 1 -- 172.21.15.32:0/438474416 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f085c014440 con 0x7f0864103540 2024-05-03T18:47:32.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:32.788+0000 7f086995a700 1 -- 172.21.15.32:0/438474416 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f086419a470 con 0x7f0864103540 2024-05-03T18:47:32.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:32.788+0000 7f0868958700 1 -- 172.21.15.32:0/438474416 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f085c00ce60 con 0x7f0864103540 2024-05-03T18:47:32.792 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:32.788+0000 7f086995a700 1 -- 172.21.15.32:0/438474416 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08641a6730 con 0x7f0864103540 2024-05-03T18:47:32.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:32.789+0000 7f086995a700 1 -- 172.21.15.32:0/438474416 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f086410ba70 con 0x7f0864103540 2024-05-03T18:47:32.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:32.790+0000 7f0868958700 1 -- 172.21.15.32:0/438474416 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f085c01c070 con 0x7f0864103540 2024-05-03T18:47:32.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:32.791+0000 7f0868958700 1 --2- 172.21.15.32:0/438474416 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f08500b2910 0x7f08500b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:32.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:32.791+0000 7f08627fc700 1 --2- 172.21.15.32:0/438474416 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f08500b2910 0x7f08500b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:32.793 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:32.791+0000 7f0868958700 1 -- 172.21.15.32:0/438474416 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8356+0+0 (secure 0 0 0) 0x7f085c0e1c30 con 0x7f0864103540 2024-05-03T18:47:32.794 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:32.791+0000 7f08627fc700 1 --2- 172.21.15.32:0/438474416 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f08500b2910 0x7f08500b4dd0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f086407ac50 tx=0x7f084c009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:47:32.795 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:32.794+0000 7f0868958700 1 -- 172.21.15.32:0/438474416 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f085c0a9820 con 0x7f0864103540 2024-05-03T18:47:33.016 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:33.015+0000 7f086995a700 1 -- 172.21.15.32:0/438474416 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f086419af50 con 0x7f0864103540 2024-05-03T18:47:33.018 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:33.017+0000 7f0868958700 1 -- 172.21.15.32:0/438474416 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+22814 (secure 0 0 0) 0x7f085c0a9120 con 0x7f0864103540 2024-05-03T18:47:33.018 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-03T18:47:33.018 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":82,"fsid":"f500455e-097b-11ef-bc97-c7b262605968","created":"2024-05-03T18:36:43.750772+0000","modified":"2024-05-03T18:46:47.787519+0000","last_up_change":"2024-05-03T18:43:50.933818+0000","last_in_change":"2024-05-03T18:43:31.804386+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-03T18:40:07.981853+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-03T18:44:44.607622+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-03T18:44:48.077391+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0430c3c0-2c50-471c-9c74-2869a0627c74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":262324956},{"type":"v1","addr":"172.21.15.32:6803","nonce":262324956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":262324956},{"type":"v1","addr":"172.21.15.32:6805","nonce":262324956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":262324956},{"type":"v1","addr":"172.21.15.32:6809","nonce":262324956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":262324956},{"type":"v1","addr":"172.21.15.32:6807","nonce":262324956}]},"public_addr":"172.21.15.32:6803/262324956","cluster_addr":"172.21.15.32:6805/262324956","heartbeat_back_addr":"172.21.15.32:6809/262324956","heartbeat_front_addr":"172.21.15.32:6807/262324956","state":["exists","up"]},{"osd":1,"uuid":"ba8f0b67-1d7d-4d4c-8276-eff664d21f02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1209907302},{"type":"v1","addr":"172.21.15.45:6801","nonce":1209907302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1209907302},{"type":"v1","addr":"172.21.15.45:6803","nonce":1209907302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1209907302},{"type":"v1","addr":"172.21.15.45:6807","nonce":1209907302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1209907302},{"type":"v1","addr":"172.21.15.45:6805","nonce":1209907302}]},"public_addr":"172.21.15.45:6801/1209907302","cluster_addr":"172.21.15.45:6803/1209907302","heartbeat_back_addr":"172.21.15.45:6807/1209907302","heartbeat_front_addr":"172.21.15.45:6805/1209907302","state":["exists","up"]},{"osd":2,"uuid":"5624eabc-be47-4bb8-9d8e-b84734769c83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":2308128926},{"type":"v1","addr":"172.21.15.113:6801","nonce":2308128926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":2308128926},{"type":"v1","addr":"172.21.15.113:6803","nonce":2308128926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":2308128926},{"type":"v1","addr":"172.21.15.113:6807","nonce":2308128926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":2308128926},{"type":"v1","addr":"172.21.15.113:6805","nonce":2308128926}]},"public_addr":"172.21.15.113:6801/2308128926","cluster_addr":"172.21.15.113:6803/2308128926","heartbeat_back_addr":"172.21.15.113:6807/2308128926","heartbeat_front_addr":"172.21.15.113:6805/2308128926","state":["exists","up"]},{"osd":3,"uuid":"62c9c977-afe1-4112-beba-b25af7caa2f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":2606514063},{"type":"v1","addr":"172.21.15.32:6811","nonce":2606514063}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":2606514063},{"type":"v1","addr":"172.21.15.32:6813","nonce":2606514063}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":2606514063},{"type":"v1","addr":"172.21.15.32:6817","nonce":2606514063}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":2606514063},{"type":"v1","addr":"172.21.15.32:6815","nonce":2606514063}]},"public_addr":"172.21.15.32:6811/2606514063","cluster_addr":"172.21.15.32:6813/2606514063","heartbeat_back_addr":"172.21.15.32:6817/2606514063","heartbeat_front_addr":"172.21.15.32:6815/2606514063","state":["exists","up"]},{"osd":4,"uuid":"99ad449f-b0bc-4aa1-b0cf-4a7a458126e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":71932118},{"type":"v1","addr":"172.21.15.45:6809","nonce":71932118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":71932118},{"type":"v1","addr":"172.21.15.45:6811","nonce":71932118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":71932118},{"type":"v1","addr":"172.21.15.45:6815","nonce":71932118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":71932118},{"type":"v1","addr":"172.21.15.45:6813","nonce":71932118}]},"public_addr":"172.21.15.45:6809/71932118","cluster_addr":"172.21.15.45:6811/71932118","heartbeat_back_addr":"172.21.15.45:6815/71932118","heartbeat_front_addr":"172.21.15.45:6813/71932118","state":["exists","up"]},{"osd":5,"uuid":"a6688ade-47d7-4a52-807f-3c880a8ffcbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":4085018597},{"type":"v1","addr":"172.21.15.113:6809","nonce":4085018597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":4085018597},{"type":"v1","addr":"172.21.15.113:6811","nonce":4085018597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":4085018597},{"type":"v1","addr":"172.21.15.113:6815","nonce":4085018597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":4085018597},{"type":"v1","addr":"172.21.15.113:6813","nonce":4085018597}]},"public_addr":"172.21.15.113:6809/4085018597","cluster_addr":"172.21.15.113:6811/4085018597","heartbeat_back_addr":"172.21.15.113:6815/4085018597","heartbeat_front_addr":"172.21.15.113:6813/4085018597","state":["exists","up"]},{"osd":6,"uuid":"1d9ebda9-55c7-473a-9dc0-fc60845bbfe9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":791407796},{"type":"v1","addr":"172.21.15.32:6819","nonce":791407796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":791407796},{"type":"v1","addr":"172.21.15.32:6821","nonce":791407796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":791407796},{"type":"v1","addr":"172.21.15.32:6825","nonce":791407796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":791407796},{"type":"v1","addr":"172.21.15.32:6823","nonce":791407796}]},"public_addr":"172.21.15.32:6819/791407796","cluster_addr":"172.21.15.32:6821/791407796","heartbeat_back_addr":"172.21.15.32:6825/791407796","heartbeat_front_addr":"172.21.15.32:6823/791407796","state":["exists","up"]},{"osd":7,"uuid":"988cce65-23ed-4814-ac09-177eb2f89fe1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1032572438},{"type":"v1","addr":"172.21.15.45:6817","nonce":1032572438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1032572438},{"type":"v1","addr":"172.21.15.45:6819","nonce":1032572438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1032572438},{"type":"v1","addr":"172.21.15.45:6823","nonce":1032572438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1032572438},{"type":"v1","addr":"172.21.15.45:6821","nonce":1032572438}]},"public_addr":"172.21.15.45:6817/1032572438","cluster_addr":"172.21.15.45:6819/1032572438","heartbeat_back_addr":"172.21.15.45:6823/1032572438","heartbeat_front_addr":"172.21.15.45:6821/1032572438","state":["exists","up"]},{"osd":8,"uuid":"816e8ce8-5afa-4b52-89be-f7793ed12a0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3569181558},{"type":"v1","addr":"172.21.15.113:6817","nonce":3569181558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3569181558},{"type":"v1","addr":"172.21.15.113:6819","nonce":3569181558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3569181558},{"type":"v1","addr":"172.21.15.113:6823","nonce":3569181558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3569181558},{"type":"v1","addr":"172.21.15.113:6821","nonce":3569181558}]},"public_addr":"172.21.15.113:6817/3569181558","cluster_addr":"172.21.15.113:6819/3569181558","heartbeat_back_addr":"172.21.15.113:6823/3569181558","heartbeat_front_addr":"172.21.15.113:6821/3569181558","state":["exists","up"]},{"osd":9,"uuid":"1d35c435-36b8-40e8-a2d4-0efe904f1107","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":4290196345},{"type":"v1","addr":"172.21.15.32:6827","nonce":4290196345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":4290196345},{"type":"v1","addr":"172.21.15.32:6829","nonce":4290196345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":4290196345},{"type":"v1","addr":"172.21.15.32:6833","nonce":4290196345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":4290196345},{"type":"v1","addr":"172.21.15.32:6831","nonce":4290196345}]},"public_addr":"172.21.15.32:6827/4290196345","cluster_addr":"172.21.15.32:6829/4290196345","heartbeat_back_addr":"172.21.15.32:6833/4290196345","heartbeat_front_addr":"172.21.15.32:6831/4290196345","state":["exists","up"]},{"osd":10,"uuid":"9af7879e-aa8d-4ef8-b289-4b145b614820","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1817064361},{"type":"v1","addr":"172.21.15.45:6825","nonce":1817064361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1817064361},{"type":"v1","addr":"172.21.15.45:6827","nonce":1817064361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1817064361},{"type":"v1","addr":"172.21.15.45:6831","nonce":1817064361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1817064361},{"type":"v1","addr":"172.21.15.45:6829","nonce":1817064361}]},"public_addr":"172.21.15.45:6825/1817064361","cluster_addr":"172.21.15.45:6827/1817064361","heartbeat_back_addr":"172.21.15.45:6831/1817064361","heartbeat_front_addr":"172.21.15.45:6829/1817064361","state":["exists","up"]},{"osd":11,"uuid":"22981709-1f9d-46f6-892a-5678394a2e1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":703464306},{"type":"v1","addr":"172.21.15.113:6825","nonce":703464306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":703464306},{"type":"v1","addr":"172.21.15.113:6827","nonce":703464306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":703464306},{"type":"v1","addr":"172.21.15.113:6831","nonce":703464306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":703464306},{"type":"v1","addr":"172.21.15.113:6829","nonce":703464306}]},"public_addr":"172.21.15.113:6825/703464306","cluster_addr":"172.21.15.113:6827/703464306","heartbeat_back_addr":"172.21.15.113:6831/703464306","heartbeat_front_addr":"172.21.15.113:6829/703464306","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:39:18.455575+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:39:42.234466+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:40:05.843072+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:40:30.822533+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:40:54.952487+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:41:18.750184+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:41:43.403326+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:42:08.285730+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:42:32.861740+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:42:59.307277+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:43:24.509691+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:43:49.850077+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":2,"to":1}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.c","mappings":[{"from":5,"to":8}]},{"pgid":"3.e","mappings":[{"from":4,"to":7}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.2f","mappings":[{"from":1,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"3.3b","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:0/743758274":"2024-05-04T18:37:45.879602+0000","172.21.15.32:6801/2019702098":"2024-05-04T18:37:45.879602+0000","172.21.15.32:6800/2019702098":"2024-05-04T18:37:45.879602+0000","172.21.15.32:0/3192385439":"2024-05-04T18:37:18.492319+0000","172.21.15.32:0/1450308494":"2024-05-04T18:37:18.492319+0000","172.21.15.32:0/3957699240":"2024-05-04T18:37:45.879602+0000","172.21.15.32:0/3310503393":"2024-05-04T18:37:18.492319+0000","172.21.15.32:6801/3945405684":"2024-05-04T18:37:18.492319+0000","172.21.15.32:6800/3945405684":"2024-05-04T18:37:18.492319+0000","172.21.15.32:0/2471164468":"2024-05-04T18:37:45.879602+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-03T18:47:33.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:33.019+0000 7f086995a700 1 -- 172.21.15.32:0/438474416 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f08500b2910 msgr2=0x7f08500b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:33.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:33.019+0000 7f086995a700 1 --2- 172.21.15.32:0/438474416 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f08500b2910 0x7f08500b4dd0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f086407ac50 tx=0x7f084c009190 comp rx=0 tx=0).stop 2024-05-03T18:47:33.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:33.019+0000 7f086995a700 1 -- 172.21.15.32:0/438474416 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0864103540 msgr2=0x7f086419f860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:33.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:33.019+0000 7f086995a700 1 --2- 172.21.15.32:0/438474416 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0864103540 0x7f086419f860 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f085c00ba50 tx=0x7f085c00bd60 comp rx=0 tx=0).stop 2024-05-03T18:47:33.020 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:33.019+0000 7f086995a700 1 -- 172.21.15.32:0/438474416 shutdown_connections 2024-05-03T18:47:33.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:33.019+0000 7f086995a700 1 --2- 172.21.15.32:0/438474416 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f08500b2910 0x7f08500b4dd0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:33.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:33.019+0000 7f086995a700 1 --2- 172.21.15.32:0/438474416 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08641095e0 0x7f0864199a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:33.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:33.019+0000 7f086995a700 1 --2- 172.21.15.32:0/438474416 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0864103540 0x7f086419f860 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:33.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:33.019+0000 7f086995a700 1 --2- 172.21.15.32:0/438474416 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0864103f00 0x7f086419fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:33.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:33.019+0000 7f086995a700 1 -- 172.21.15.32:0/438474416 >> 172.21.15.32:0/438474416 conn(0x7f086407a650 msgr2=0x7f0864102000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:47:33.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:33.019+0000 7f086995a700 1 -- 172.21.15.32:0/438474416 shutdown_connections 2024-05-03T18:47:33.021 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:33.019+0000 7f086995a700 1 -- 172.21.15.32:0/438474416 wait complete. 2024-05-03T18:47:33.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:47:33 smithi032 ceph-mon[29763]: pgmap v365: 97 pgs: 97 active+clean; 631 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:47:33.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:47:33 smithi032 ceph-mon[29763]: from='client.? 172.21.15.32:0/438474416' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T18:47:33.735 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 25, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-05-03T18:44:51.155050+0000', 'modified': '2024-05-03T18:45:08.404530+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24478, 'mds_1': 24469, 'mds_2': 24463}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24425': {'gid': 24425, 'name': 'h', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.45:6837/428055392', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6836', 'nonce': 428055392}, {'type': 'v1', 'addr': '172.21.15.45:6837', 'nonce': 428055392}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24430': {'gid': 24430, 'name': 'l', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.113:6833/3323558791', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.113:6832', 'nonce': 3323558791}, {'type': 'v1', 'addr': '172.21.15.113:6833', 'nonce': 3323558791}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24446': {'gid': 24446, 'name': 'b', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.45:6839/970804040', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6838', 'nonce': 970804040}, {'type': 'v1', 'addr': '172.21.15.45:6839', 'nonce': 970804040}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24463': {'gid': 24463, 'name': 'f', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.113:6835/2588738276', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.113:6834', 'nonce': 2588738276}, {'type': 'v1', 'addr': '172.21.15.113:6835', 'nonce': 2588738276}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24469': {'gid': 24469, 'name': 'c', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.113:6837/3677284460', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.113:6836', 'nonce': 3677284460}, {'type': 'v1', 'addr': '172.21.15.113:6837', 'nonce': 3677284460}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24478': {'gid': 24478, 'name': 'i', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.113:6839/942050693', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.113:6838', 'nonce': 942050693}, {'type': 'v1', 'addr': '172.21.15.113:6839', 'nonce': 942050693}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24478, 'qdb_cluster': [24469, 24478, 24463]} 2024-05-03T18:47:33.735 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-05-03T18:47:33.735 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid f500455e-097b-11ef-bc97-c7b262605968 -- ceph tell mds.f status 2024-05-03T18:47:33.747 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:47:33 smithi045 ceph-mon[33299]: pgmap v365: 97 pgs: 97 active+clean; 631 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:47:33.747 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:47:33 smithi045 ceph-mon[33299]: from='client.? 172.21.15.32:0/438474416' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T18:47:33.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:47:33 smithi113 ceph-mon[33540]: pgmap v365: 97 pgs: 97 active+clean; 631 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:47:33.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:47:33 smithi113 ceph-mon[33540]: from='client.? 172.21.15.32:0/438474416' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T18:47:33.990 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:47:35.344 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:47:35 smithi032 ceph-mon[29763]: pgmap v366: 97 pgs: 97 active+clean; 631 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:47:35.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.695+0000 7f3f94b64700 1 -- 172.21.15.32:0/401501552 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f90108850 msgr2=0x7f3f901103d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:35.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.695+0000 7f3f94b64700 1 --2- 172.21.15.32:0/401501552 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f90108850 0x7f3f901103d0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f3f8800b0d0 tx=0x7f3f8800b3e0 comp rx=0 tx=0).stop 2024-05-03T18:47:35.696 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.695+0000 7f3f94b64700 1 -- 172.21.15.32:0/401501552 shutdown_connections 2024-05-03T18:47:35.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.695+0000 7f3f94b64700 1 --2- 172.21.15.32:0/401501552 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f90108850 0x7f3f901103d0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:35.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.695+0000 7f3f94b64700 1 --2- 172.21.15.32:0/401501552 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3f90103750 0x7f3f90103b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:35.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.695+0000 7f3f94b64700 1 --2- 172.21.15.32:0/401501552 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f90104100 0x7f3f90108120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:35.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.695+0000 7f3f94b64700 1 -- 172.21.15.32:0/401501552 >> 172.21.15.32:0/401501552 conn(0x7f3f9007a600 msgr2=0x7f3f9007aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:47:35.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.695+0000 7f3f94b64700 1 -- 172.21.15.32:0/401501552 shutdown_connections 2024-05-03T18:47:35.697 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.695+0000 7f3f94b64700 1 -- 172.21.15.32:0/401501552 wait complete. 2024-05-03T18:47:35.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.696+0000 7f3f94b64700 1 Processor -- start 2024-05-03T18:47:35.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.696+0000 7f3f94b64700 1 -- start start 2024-05-03T18:47:35.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.696+0000 7f3f94b64700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f90103750 0x7f3f9019bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:35.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.696+0000 7f3f94b64700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3f90104100 0x7f3f9019c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:35.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.696+0000 7f3f94b64700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f90108850 0x7f3f901a0600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:35.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.696+0000 7f3f94b64700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f3f901136a0 con 0x7f3f90108850 2024-05-03T18:47:35.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.696+0000 7f3f94b64700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3f90113520 con 0x7f3f90103750 2024-05-03T18:47:35.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.696+0000 7f3f94b64700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f3f90113820 con 0x7f3f90104100 2024-05-03T18:47:35.699 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.696+0000 7f3f8ed9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f90108850 0x7f3f901a0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:35.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.696+0000 7f3f8ed9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f90108850 0x7f3f901a0600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34360/0 (socket says 172.21.15.32:34360) 2024-05-03T18:47:35.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.696+0000 7f3f8ed9d700 1 -- 172.21.15.32:0/58269737 learned_addr learned my addr 172.21.15.32:0/58269737 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:47:35.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.696+0000 7f3f8e59c700 1 --2- 172.21.15.32:0/58269737 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f90103750 0x7f3f9019bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:35.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.696+0000 7f3f8dd9b700 1 --2- 172.21.15.32:0/58269737 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3f90104100 0x7f3f9019c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:35.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.696+0000 7f3f8ed9d700 1 -- 172.21.15.32:0/58269737 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3f90104100 msgr2=0x7f3f9019c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:35.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.696+0000 7f3f8ed9d700 1 --2- 172.21.15.32:0/58269737 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3f90104100 0x7f3f9019c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:35.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.696+0000 7f3f8ed9d700 1 -- 172.21.15.32:0/58269737 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f90103750 msgr2=0x7f3f9019bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:35.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.696+0000 7f3f8ed9d700 1 --2- 172.21.15.32:0/58269737 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f90103750 0x7f3f9019bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:35.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.696+0000 7f3f8ed9d700 1 -- 172.21.15.32:0/58269737 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f901a0d40 con 0x7f3f90108850 2024-05-03T18:47:35.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.696+0000 7f3f8e59c700 1 --2- 172.21.15.32:0/58269737 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f90103750 0x7f3f9019bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T18:47:35.701 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.696+0000 7f3f8dd9b700 1 --2- 172.21.15.32:0/58269737 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3f90104100 0x7f3f9019c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:47:35.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.697+0000 7f3f8ed9d700 1 --2- 172.21.15.32:0/58269737 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f90108850 0x7f3f901a0600 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f3f880095c0 tx=0x7f3f88007ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:47:35.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.697+0000 7f3f7f7fe700 1 -- 172.21.15.32:0/58269737 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f8801f070 con 0x7f3f90108850 2024-05-03T18:47:35.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.697+0000 7f3f94b64700 1 -- 172.21.15.32:0/58269737 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f901a0fd0 con 0x7f3f90108850 2024-05-03T18:47:35.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.697+0000 7f3f7f7fe700 1 -- 172.21.15.32:0/58269737 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f88003c00 con 0x7f3f90108850 2024-05-03T18:47:35.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.697+0000 7f3f7f7fe700 1 -- 172.21.15.32:0/58269737 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f8801b6a0 con 0x7f3f90108850 2024-05-03T18:47:35.702 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.697+0000 7f3f94b64700 1 -- 172.21.15.32:0/58269737 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f901a88b0 con 0x7f3f90108850 2024-05-03T18:47:35.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.699+0000 7f3f7f7fe700 1 -- 172.21.15.32:0/58269737 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f3f88022070 con 0x7f3f90108850 2024-05-03T18:47:35.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.700+0000 7f3f7f7fe700 1 --2- 172.21.15.32:0/58269737 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f3f780b2910 0x7f3f780b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:35.708 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.700+0000 7f3f8e59c700 1 --2- 172.21.15.32:0/58269737 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f3f780b2910 0x7f3f780b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:35.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.700+0000 7f3f7f7fe700 1 -- 172.21.15.32:0/58269737 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8356+0+0 (secure 0 0 0) 0x7f3f8800f020 con 0x7f3f90108850 2024-05-03T18:47:35.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.700+0000 7f3f8e59c700 1 --2- 172.21.15.32:0/58269737 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f3f780b2910 0x7f3f780b4dd0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f3f800099d0 tx=0x7f3f80008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:47:35.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.701+0000 7f3f9f3f7b80 1 Processor -- start 2024-05-03T18:47:35.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.701+0000 7f3f9f3f7b80 1 -- start start 2024-05-03T18:47:35.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.701+0000 7f3f9f3f7b80 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b901b96c10 0x55b901b96ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:35.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.701+0000 7f3f9f3f7b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b901b97530 0x55b901b9acf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:35.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.701+0000 7f3f9f3f7b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b901b9b230 0x55b901b9b690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:35.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.701+0000 7f3f9f3f7b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55b901ae9810 con 0x55b901b97530 2024-05-03T18:47:35.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.701+0000 7f3f9f3f7b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55b901a88770 con 0x55b901b9b230 2024-05-03T18:47:35.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.701+0000 7f3f9f3f7b80 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55b901ae57d0 con 0x55b901b96c10 2024-05-03T18:47:35.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.701+0000 7f3f8dd9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b901b97530 0x55b901b9acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:35.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.701+0000 7f3f8dd9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b901b97530 0x55b901b9acf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34370/0 (socket says 172.21.15.32:34370) 2024-05-03T18:47:35.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.701+0000 7f3f8dd9b700 1 -- 172.21.15.32:0/939955357 learned_addr learned my addr 172.21.15.32:0/939955357 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:47:35.709 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.702+0000 7f3f8ed9d700 1 --2- 172.21.15.32:0/939955357 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b901b9b230 0x55b901b9b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:35.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.702+0000 7f3f8e59c700 1 --2- 172.21.15.32:0/939955357 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b901b96c10 0x55b901b96ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:35.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.702+0000 7f3f8dd9b700 1 -- 172.21.15.32:0/939955357 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b901b96c10 msgr2=0x55b901b96ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:47:35.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.702+0000 7f3f8dd9b700 1 --2- 172.21.15.32:0/939955357 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b901b96c10 0x55b901b96ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:35.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.702+0000 7f3f8dd9b700 1 -- 172.21.15.32:0/939955357 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b901b9b230 msgr2=0x55b901b9b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:47:35.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.702+0000 7f3f8dd9b700 1 --2- 172.21.15.32:0/939955357 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b901b9b230 0x55b901b9b690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:35.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.702+0000 7f3f8dd9b700 1 -- 172.21.15.32:0/939955357 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b901a36880 con 0x55b901b97530 2024-05-03T18:47:35.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.702+0000 7f3f8dd9b700 1 --2- 172.21.15.32:0/939955357 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b901b97530 0x55b901b9acf0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f3f84009cb0 tx=0x7f3f8400e2f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a28ec8660df8ba4 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:47:35.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.702+0000 7f3f7cff9700 1 -- 172.21.15.32:0/939955357 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f84016070 con 0x55b901b97530 2024-05-03T18:47:35.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.702+0000 7f3f7cff9700 1 -- 172.21.15.32:0/939955357 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f84004830 con 0x55b901b97530 2024-05-03T18:47:35.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.702+0000 7f3f7cff9700 1 -- 172.21.15.32:0/939955357 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f84005480 con 0x55b901b97530 2024-05-03T18:47:35.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.703+0000 7f3f9f3f7b80 1 -- 172.21.15.32:0/939955357 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b901b97530 msgr2=0x55b901b9acf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:35.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.703+0000 7f3f9f3f7b80 1 --2- 172.21.15.32:0/939955357 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b901b97530 0x55b901b9acf0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f3f84009cb0 tx=0x7f3f8400e2f0 comp rx=0 tx=0).stop 2024-05-03T18:47:35.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.703+0000 7f3f9f3f7b80 1 -- 172.21.15.32:0/939955357 shutdown_connections 2024-05-03T18:47:35.710 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.703+0000 7f3f9f3f7b80 1 --2- 172.21.15.32:0/939955357 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b901b9b230 0x55b901b9b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:35.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.703+0000 7f3f9f3f7b80 1 --2- 172.21.15.32:0/939955357 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b901b96c10 0x55b901b96ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:35.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.703+0000 7f3f9f3f7b80 1 --2- 172.21.15.32:0/939955357 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b901b97530 0x55b901b9acf0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:35.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.703+0000 7f3f9f3f7b80 1 -- 172.21.15.32:0/939955357 >> 172.21.15.32:0/939955357 conn(0x55b901ac30b0 msgr2=0x55b901b91280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:47:35.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.703+0000 7f3f9f3f7b80 1 -- 172.21.15.32:0/939955357 shutdown_connections 2024-05-03T18:47:35.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.703+0000 7f3f9f3f7b80 1 -- 172.21.15.32:0/939955357 wait complete. 2024-05-03T18:47:35.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.703+0000 7f3f9f3f7b80 1 -- start start 2024-05-03T18:47:35.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.703+0000 7f3f9f3f7b80 1 Processor -- start 2024-05-03T18:47:35.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.704+0000 7f3f9f3f7b80 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b901b96c10 0x55b901ba26b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:35.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.704+0000 7f3f9f3f7b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b901b97530 0x55b901b9fa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:35.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.704+0000 7f3f9f3f7b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b901b9b230 0x55b901b9ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:35.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.704+0000 7f3f9f3f7b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55b901ba3490 con 0x55b901b97530 2024-05-03T18:47:35.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.704+0000 7f3f9f3f7b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55b901a4fbe0 con 0x55b901b9b230 2024-05-03T18:47:35.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.704+0000 7f3f9f3f7b80 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55b901ba3610 con 0x55b901b96c10 2024-05-03T18:47:35.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.704+0000 7f3f9f3f7b80 10 client.0 _ll_register_callbacks cb 0x55b901a818d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:47:35.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.704+0000 7f3f8dd9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b901b97530 0x55b901b9fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:35.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.704+0000 7f3f8dd9b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b901b97530 0x55b901b9fa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34386/0 (socket says 172.21.15.32:34386) 2024-05-03T18:47:35.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.704+0000 7f3f8dd9b700 1 -- 172.21.15.32:0/184029108 learned_addr learned my addr 172.21.15.32:0/184029108 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:47:35.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.704+0000 7f3f8ed9d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b901b9b230 0x55b901b9ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:35.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.704+0000 7f3f8e59c700 1 --2- 172.21.15.32:0/184029108 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b901b96c10 0x55b901ba26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:35.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.704+0000 7f3f8dd9b700 1 -- 172.21.15.32:0/184029108 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b901b96c10 msgr2=0x55b901ba26b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:47:35.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.704+0000 7f3f8dd9b700 1 --2- 172.21.15.32:0/184029108 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b901b96c10 0x55b901ba26b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:35.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.704+0000 7f3f8dd9b700 1 -- 172.21.15.32:0/184029108 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b901b9b230 msgr2=0x55b901b9ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:47:35.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.704+0000 7f3f8dd9b700 1 --2- 172.21.15.32:0/184029108 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b901b9b230 0x55b901b9ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:35.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.704+0000 7f3f8dd9b700 1 -- 172.21.15.32:0/184029108 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b901a85640 con 0x55b901b97530 2024-05-03T18:47:35.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.704+0000 7f3f8dd9b700 1 --2- 172.21.15.32:0/184029108 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b901b97530 0x55b901b9fa50 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f3f8400ebd0 tx=0x7f3f8400ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ae2a99aa251702a server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:47:35.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.704+0000 7f3f75ffb700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-05-03T18:47:35.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.704+0000 7f3f75ffb700 1 -- 172.21.15.32:0/184029108 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f84028060 con 0x55b901b97530 2024-05-03T18:47:35.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.705+0000 7f3f75ffb700 1 -- 172.21.15.32:0/184029108 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8356+0+0 (secure 0 0 0) 0x7f3f840052d0 con 0x55b901b97530 2024-05-03T18:47:35.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.705+0000 7f3f9f3f7b80 1 -- 172.21.15.32:0/184029108 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3f84004830 con 0x55b901b97530 2024-05-03T18:47:35.712 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.705+0000 7f3f75ffb700 1 -- 172.21.15.32:0/184029108 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3f84016040 con 0x55b901b97530 2024-05-03T18:47:35.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.705+0000 7f3f9f3f7b80 10 client.15090 fetch_fsmap learned FSMap version 25 2024-05-03T18:47:35.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.705+0000 7f3f9f3f7b80 1 -- 172.21.15.32:0/184029108 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b901badcd0 con 0x55b901b97530 2024-05-03T18:47:35.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.705+0000 7f3f75ffb700 1 -- 172.21.15.32:0/184029108 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f84004830 con 0x55b901b97530 2024-05-03T18:47:35.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.705+0000 7f3f75ffb700 1 -- 172.21.15.32:0/184029108 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f3f8400f040 con 0x55b901b97530 2024-05-03T18:47:35.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.705+0000 7f3f9f3f7b80 10 client.15090 fetch_fsmap finished waiting for FSMap version 25 2024-05-03T18:47:35.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.705+0000 7f3f9f3f7b80 10 client.15090 resolve_mds: resolved name 'f' to daemon mds.f 2024-05-03T18:47:35.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.705+0000 7f3f9f3f7b80 20 client.15090 populate_metadata read hostname 'smithi032' 2024-05-03T18:47:35.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.705+0000 7f3f9f3f7b80 1 --2- 172.21.15.32:0/184029108 >> [v2:172.21.15.113:6834/2588738276,v1:172.21.15.113:6835/2588738276] conn(0x55b901baff90 0x55b901bb1400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:35.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.705+0000 7f3f9f3f7b80 4 client.15090 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-03T18:47:35.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.705+0000 7f3f9f3f7b80 1 -- 172.21.15.32:0/184029108 --> [v2:172.21.15.113:6834/2588738276,v1:172.21.15.113:6835/2588738276] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b901af71f0 con 0x55b901baff90 2024-05-03T18:47:35.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.706+0000 7f3f8e59c700 1 --2- 172.21.15.32:0/184029108 >> [v2:172.21.15.113:6834/2588738276,v1:172.21.15.113:6835/2588738276] conn(0x55b901baff90 0x55b901bb1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:35.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.706+0000 7f3f8e59c700 1 --2- 172.21.15.32:0/184029108 >> [v2:172.21.15.113:6834/2588738276,v1:172.21.15.113:6835/2588738276] conn(0x55b901baff90 0x55b901bb1400 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fb4d2c15a1e1d445 server_cookie=76ed8525ebccb138 in_seq=0 out_seq=0 2024-05-03T18:47:35.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.706+0000 7f3f75ffb700 10 client.15090 ms_handle_connect on v2:172.21.15.113:6834/2588738276 2024-05-03T18:47:35.713 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.707+0000 7f3f75ffb700 1 -- 172.21.15.32:0/184029108 <== mds.2 v2:172.21.15.113:6834/2588738276 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b901af71f0 con 0x55b901baff90 2024-05-03T18:47:35.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.707+0000 7f3f75ffb700 10 client.15090 handle_command_reply: tid=0 2024-05-03T18:47:35.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.707+0000 7f3f9f3f7b80 1 client.15090 shutdown 2024-05-03T18:47:35.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.707+0000 7f3f9f3f7b80 1 -- 172.21.15.32:0/184029108 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b901b97530 msgr2=0x55b901b9fa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:35.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.707+0000 7f3f9f3f7b80 1 --2- 172.21.15.32:0/184029108 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b901b97530 0x55b901b9fa50 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f3f8400ebd0 tx=0x7f3f8400ec00 comp rx=0 tx=0).stop 2024-05-03T18:47:35.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.707+0000 7f3f9f3f7b80 1 -- 172.21.15.32:0/184029108 shutdown_connections 2024-05-03T18:47:35.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.707+0000 7f3f9f3f7b80 1 --2- 172.21.15.32:0/184029108 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b901b9b230 0x55b901b9ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:35.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.707+0000 7f3f9f3f7b80 1 --2- 172.21.15.32:0/184029108 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b901b96c10 0x55b901ba26b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:35.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.707+0000 7f3f9f3f7b80 1 --2- 172.21.15.32:0/184029108 >> [v2:172.21.15.113:6834/2588738276,v1:172.21.15.113:6835/2588738276] conn(0x55b901baff90 0x55b901bb1400 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:35.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.707+0000 7f3f9f3f7b80 1 --2- 172.21.15.32:0/184029108 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b901b97530 0x55b901b9fa50 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:35.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.707+0000 7f3f9f3f7b80 1 -- 172.21.15.32:0/184029108 >> 172.21.15.32:0/184029108 conn(0x55b901ac30b0 msgr2=0x55b901b908f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:47:35.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.707+0000 7f3f75ffb700 0 client.15090 ms_handle_reset on v2:172.21.15.113:6834/2588738276 2024-05-03T18:47:35.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.707+0000 7f3f9f3f7b80 1 -- 172.21.15.32:0/184029108 shutdown_connections 2024-05-03T18:47:35.714 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.707+0000 7f3f9f3f7b80 1 -- 172.21.15.32:0/184029108 wait complete. 2024-05-03T18:47:35.715 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.707+0000 7f3f9f3f7b80 20 client.15090 trim_cache size 0 max 16384 2024-05-03T18:47:35.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.728+0000 7f3f9f3f7b80 1 Processor -- start 2024-05-03T18:47:35.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.728+0000 7f3f9f3f7b80 1 -- start start 2024-05-03T18:47:35.731 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.729+0000 7f3f9f3f7b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b901b96c10 0x55b901b9a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:35.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.729+0000 7f3f9f3f7b80 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b901b97530 0x55b901b91070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:35.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.729+0000 7f3f9f3f7b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b901b9b230 0x55b901b915b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:35.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.729+0000 7f3f9f3f7b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55b901baf740 con 0x55b901b96c10 2024-05-03T18:47:35.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.729+0000 7f3f9f3f7b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55b901bafe10 con 0x55b901b9b230 2024-05-03T18:47:35.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.729+0000 7f3f9f3f7b80 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55b901bb04e0 con 0x55b901b97530 2024-05-03T18:47:35.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.729+0000 7f3f8e59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b901b96c10 0x55b901b9a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:35.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.729+0000 7f3f8e59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b901b96c10 0x55b901b9a370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34388/0 (socket says 172.21.15.32:34388) 2024-05-03T18:47:35.732 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.729+0000 7f3f8e59c700 1 -- 172.21.15.32:0/1333019084 learned_addr learned my addr 172.21.15.32:0/1333019084 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:47:35.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.729+0000 7f3f8ed9d700 1 --2- 172.21.15.32:0/1333019084 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b901b9b230 0x55b901b915b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:35.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.729+0000 7f3f8dd9b700 1 --2- 172.21.15.32:0/1333019084 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b901b97530 0x55b901b91070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:35.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.729+0000 7f3f8e59c700 1 -- 172.21.15.32:0/1333019084 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b901b97530 msgr2=0x55b901b91070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:47:35.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.729+0000 7f3f8e59c700 1 --2- 172.21.15.32:0/1333019084 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b901b97530 0x55b901b91070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:35.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.729+0000 7f3f8e59c700 1 -- 172.21.15.32:0/1333019084 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b901b9b230 msgr2=0x55b901b915b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:47:35.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.729+0000 7f3f8e59c700 1 --2- 172.21.15.32:0/1333019084 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b901b9b230 0x55b901b915b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:35.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.729+0000 7f3f8e59c700 1 -- 172.21.15.32:0/1333019084 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b901bb09d0 con 0x55b901b96c10 2024-05-03T18:47:35.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.729+0000 7f3f8dd9b700 1 --2- 172.21.15.32:0/1333019084 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b901b97530 0x55b901b91070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:47:35.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.729+0000 7f3f8e59c700 1 --2- 172.21.15.32:0/1333019084 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b901b96c10 0x55b901b9a370 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f3f80008010 tx=0x7f3f80013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d85374613cc2259 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:47:35.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.729+0000 7f3f77fff700 1 -- 172.21.15.32:0/1333019084 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f80027210 con 0x55b901b96c10 2024-05-03T18:47:35.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.729+0000 7f3f77fff700 1 -- 172.21.15.32:0/1333019084 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f80027860 con 0x55b901b96c10 2024-05-03T18:47:35.733 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.729+0000 7f3f77fff700 1 -- 172.21.15.32:0/1333019084 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f80030a20 con 0x55b901b96c10 2024-05-03T18:47:35.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.730+0000 7f3f9f3f7b80 1 -- 172.21.15.32:0/1333019084 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b901b96c10 msgr2=0x55b901b9a370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:35.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.730+0000 7f3f9f3f7b80 1 --2- 172.21.15.32:0/1333019084 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b901b96c10 0x55b901b9a370 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f3f80008010 tx=0x7f3f80013000 comp rx=0 tx=0).stop 2024-05-03T18:47:35.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.730+0000 7f3f9f3f7b80 1 -- 172.21.15.32:0/1333019084 shutdown_connections 2024-05-03T18:47:35.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.730+0000 7f3f9f3f7b80 1 --2- 172.21.15.32:0/1333019084 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b901b9b230 0x55b901b915b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:35.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.730+0000 7f3f9f3f7b80 1 --2- 172.21.15.32:0/1333019084 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b901b96c10 0x55b901b9a370 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:35.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.730+0000 7f3f9f3f7b80 1 --2- 172.21.15.32:0/1333019084 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b901b97530 0x55b901b91070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:35.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.730+0000 7f3f9f3f7b80 1 -- 172.21.15.32:0/1333019084 >> 172.21.15.32:0/1333019084 conn(0x55b901ac30b0 msgr2=0x55b901b99c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:47:35.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.730+0000 7f3f9f3f7b80 1 -- 172.21.15.32:0/1333019084 shutdown_connections 2024-05-03T18:47:35.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.730+0000 7f3f9f3f7b80 1 -- 172.21.15.32:0/1333019084 wait complete. 2024-05-03T18:47:35.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.730+0000 7f3f9f3f7b80 1 -- start start 2024-05-03T18:47:35.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.730+0000 7f3f9f3f7b80 1 Processor -- start 2024-05-03T18:47:35.737 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.731+0000 7f3f9f3f7b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b901b96c10 0x55b901bbabd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:35.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.731+0000 7f3f9f3f7b80 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b901b97530 0x55b901b9c380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:35.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.731+0000 7f3f9f3f7b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b901b9b230 0x55b901b9c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:35.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.731+0000 7f3f9f3f7b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55b901bb1280 con 0x55b901b96c10 2024-05-03T18:47:35.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.731+0000 7f3f9f3f7b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55b901bb0bb0 con 0x55b901b9b230 2024-05-03T18:47:35.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.731+0000 7f3f9f3f7b80 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55b901bb1950 con 0x55b901b97530 2024-05-03T18:47:35.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.731+0000 7f3f8e59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b901b96c10 0x55b901bbabd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:35.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.731+0000 7f3f9f3f7b80 10 client.0 _ll_register_callbacks cb 0x55b901a818d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:47:35.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.731+0000 7f3f8e59c700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b901b96c10 0x55b901bbabd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34400/0 (socket says 172.21.15.32:34400) 2024-05-03T18:47:35.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.731+0000 7f3f8e59c700 1 -- 172.21.15.32:0/3162616299 learned_addr learned my addr 172.21.15.32:0/3162616299 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:47:35.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.731+0000 7f3f8ed9d700 1 --2- 172.21.15.32:0/3162616299 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b901b9b230 0x55b901b9c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:35.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.731+0000 7f3f8e59c700 1 -- 172.21.15.32:0/3162616299 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b901b97530 msgr2=0x55b901b9c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:47:35.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.731+0000 7f3f8e59c700 1 --2- 172.21.15.32:0/3162616299 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b901b97530 0x55b901b9c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:35.738 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.731+0000 7f3f8e59c700 1 -- 172.21.15.32:0/3162616299 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b901b9b230 msgr2=0x55b901b9c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:47:35.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.731+0000 7f3f8e59c700 1 --2- 172.21.15.32:0/3162616299 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b901b9b230 0x55b901b9c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:35.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.731+0000 7f3f8e59c700 1 -- 172.21.15.32:0/3162616299 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b901b9cf00 con 0x55b901b96c10 2024-05-03T18:47:35.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.731+0000 7f3f8e59c700 1 --2- 172.21.15.32:0/3162616299 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b901b96c10 0x55b901bbabd0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f3f80006d00 tx=0x7f3f80027c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=712698f3756e8d32 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:47:35.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.731+0000 7f3f75ffb700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-05-03T18:47:35.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.731+0000 7f3f75ffb700 1 -- 172.21.15.32:0/3162616299 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f80005c60 con 0x55b901b96c10 2024-05-03T18:47:35.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.732+0000 7f3f9f3f7b80 1 -- 172.21.15.32:0/3162616299 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3f80027860 con 0x55b901b96c10 2024-05-03T18:47:35.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.732+0000 7f3f75ffb700 1 -- 172.21.15.32:0/3162616299 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8356+0+0 (secure 0 0 0) 0x7f3f80005e00 con 0x55b901b96c10 2024-05-03T18:47:35.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.732+0000 7f3f75ffb700 1 -- 172.21.15.32:0/3162616299 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3f8003d020 con 0x55b901b96c10 2024-05-03T18:47:35.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.732+0000 7f3f9f3f7b80 10 client.15096 fetch_fsmap learned FSMap version 25 2024-05-03T18:47:35.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.732+0000 7f3f9f3f7b80 1 -- 172.21.15.32:0/3162616299 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b901bba730 con 0x55b901b96c10 2024-05-03T18:47:35.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.732+0000 7f3f75ffb700 1 -- 172.21.15.32:0/3162616299 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f80027860 con 0x55b901b96c10 2024-05-03T18:47:35.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.732+0000 7f3f75ffb700 1 -- 172.21.15.32:0/3162616299 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f3f80037030 con 0x55b901b96c10 2024-05-03T18:47:35.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.732+0000 7f3f9f3f7b80 10 client.15096 fetch_fsmap finished waiting for FSMap version 25 2024-05-03T18:47:35.739 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.732+0000 7f3f9f3f7b80 10 client.15096 resolve_mds: resolved name 'f' to daemon mds.f 2024-05-03T18:47:35.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.732+0000 7f3f9f3f7b80 20 client.15096 populate_metadata read hostname 'smithi032' 2024-05-03T18:47:35.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.732+0000 7f3f9f3f7b80 1 --2- 172.21.15.32:0/3162616299 >> [v2:172.21.15.113:6834/2588738276,v1:172.21.15.113:6835/2588738276] conn(0x55b901c1f9f0 0x55b901c1e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:35.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.732+0000 7f3f9f3f7b80 4 client.15096 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-03T18:47:35.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.732+0000 7f3f9f3f7b80 1 -- 172.21.15.32:0/3162616299 --> [v2:172.21.15.113:6834/2588738276,v1:172.21.15.113:6835/2588738276] -- command(tid 0: {"prefix": "status"}) -- 0x55b901af1290 con 0x55b901c1f9f0 2024-05-03T18:47:35.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.733+0000 7f3f8dd9b700 1 --2- 172.21.15.32:0/3162616299 >> [v2:172.21.15.113:6834/2588738276,v1:172.21.15.113:6835/2588738276] conn(0x55b901c1f9f0 0x55b901c1e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:35.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.733+0000 7f3f75ffb700 10 client.15096 ms_handle_connect on v2:172.21.15.113:6834/2588738276 2024-05-03T18:47:35.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.733+0000 7f3f8dd9b700 1 --2- 172.21.15.32:0/3162616299 >> [v2:172.21.15.113:6834/2588738276,v1:172.21.15.113:6835/2588738276] conn(0x55b901c1f9f0 0x55b901c1e700 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6cfb39bc8f2bde05 server_cookie=de9d6159e252a6da in_seq=0 out_seq=0 2024-05-03T18:47:35.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.733+0000 7f3f75ffb700 1 -- 172.21.15.32:0/3162616299 <== mds.2 v2:172.21.15.113:6834/2588738276 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55b901af1290 con 0x55b901c1f9f0 2024-05-03T18:47:35.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.733+0000 7f3f75ffb700 10 client.15096 handle_command_reply: tid=0 2024-05-03T18:47:35.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.733+0000 7f3f9f3f7b80 1 client.15096 shutdown 2024-05-03T18:47:35.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.733+0000 7f3f9f3f7b80 1 -- 172.21.15.32:0/3162616299 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b901b96c10 msgr2=0x55b901bbabd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:35.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.733+0000 7f3f9f3f7b80 1 --2- 172.21.15.32:0/3162616299 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b901b96c10 0x55b901bbabd0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f3f80006d00 tx=0x7f3f80027c00 comp rx=0 tx=0).stop 2024-05-03T18:47:35.740 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.733+0000 7f3f9f3f7b80 1 -- 172.21.15.32:0/3162616299 shutdown_connections 2024-05-03T18:47:35.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.733+0000 7f3f9f3f7b80 1 --2- 172.21.15.32:0/3162616299 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b901b9b230 0x55b901b9c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:35.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.733+0000 7f3f9f3f7b80 1 --2- 172.21.15.32:0/3162616299 >> [v2:172.21.15.113:6834/2588738276,v1:172.21.15.113:6835/2588738276] conn(0x55b901c1f9f0 0x55b901c1e700 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:35.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.733+0000 7f3f75ffb700 0 client.15096 ms_handle_reset on v2:172.21.15.113:6834/2588738276 2024-05-03T18:47:35.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.733+0000 7f3f9f3f7b80 1 --2- 172.21.15.32:0/3162616299 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b901b96c10 0x55b901bbabd0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:35.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.734+0000 7f3f9f3f7b80 1 --2- 172.21.15.32:0/3162616299 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b901b97530 0x55b901b9c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:35.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.734+0000 7f3f9f3f7b80 1 -- 172.21.15.32:0/3162616299 >> 172.21.15.32:0/3162616299 conn(0x55b901ac30b0 msgr2=0x55b901b9a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:47:35.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.734+0000 7f3f9f3f7b80 1 -- 172.21.15.32:0/3162616299 shutdown_connections 2024-05-03T18:47:35.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.734+0000 7f3f9f3f7b80 1 -- 172.21.15.32:0/3162616299 wait complete. 2024-05-03T18:47:35.741 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.734+0000 7f3f9f3f7b80 20 client.15096 trim_cache size 0 max 16384 2024-05-03T18:47:35.741 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-05-03T18:47:35.742 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_fsid": "f500455e-097b-11ef-bc97-c7b262605968", 2024-05-03T18:47:35.742 INFO:teuthology.orchestra.run.smithi032.stdout: "whoami": 2, 2024-05-03T18:47:35.742 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 24463, 2024-05-03T18:47:35.742 INFO:teuthology.orchestra.run.smithi032.stdout: "want_state": "up:active", 2024-05-03T18:47:35.742 INFO:teuthology.orchestra.run.smithi032.stdout: "state": "up:active", 2024-05-03T18:47:35.742 INFO:teuthology.orchestra.run.smithi032.stdout: "fs_name": "cephfs", 2024-05-03T18:47:35.742 INFO:teuthology.orchestra.run.smithi032.stdout: "rank_uptime": 152.26146728800001, 2024-05-03T18:47:35.742 INFO:teuthology.orchestra.run.smithi032.stdout: "mdsmap_epoch": 25, 2024-05-03T18:47:35.742 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch": 82, 2024-05-03T18:47:35.742 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch_barrier": 78, 2024-05-03T18:47:35.742 INFO:teuthology.orchestra.run.smithi032.stdout: "uptime": 196.16900021999999 2024-05-03T18:47:35.743 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-05-03T18:47:35.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.734+0000 7f3f94b64700 1 -- 172.21.15.32:0/58269737 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f3f780b2910 msgr2=0x7f3f780b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:35.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.734+0000 7f3f94b64700 1 --2- 172.21.15.32:0/58269737 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f3f780b2910 0x7f3f780b4dd0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f3f800099d0 tx=0x7f3f80008040 comp rx=0 tx=0).stop 2024-05-03T18:47:35.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.734+0000 7f3f94b64700 1 -- 172.21.15.32:0/58269737 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f90108850 msgr2=0x7f3f901a0600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:35.743 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.734+0000 7f3f94b64700 1 --2- 172.21.15.32:0/58269737 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f90108850 0x7f3f901a0600 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f3f880095c0 tx=0x7f3f88007ad0 comp rx=0 tx=0).stop 2024-05-03T18:47:35.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.734+0000 7f3f94b64700 1 -- 172.21.15.32:0/58269737 shutdown_connections 2024-05-03T18:47:35.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.734+0000 7f3f94b64700 1 --2- 172.21.15.32:0/58269737 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f3f780b2910 0x7f3f780b4dd0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:35.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.734+0000 7f3f94b64700 1 --2- 172.21.15.32:0/58269737 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f90108850 0x7f3f901a0600 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:35.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.734+0000 7f3f94b64700 1 --2- 172.21.15.32:0/58269737 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3f90103750 0x7f3f9019bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:35.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.734+0000 7f3f94b64700 1 --2- 172.21.15.32:0/58269737 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3f90104100 0x7f3f9019c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:35.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.734+0000 7f3f94b64700 1 -- 172.21.15.32:0/58269737 >> 172.21.15.32:0/58269737 conn(0x7f3f9007a600 msgr2=0x7f3f90101790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:47:35.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.734+0000 7f3f94b64700 1 -- 172.21.15.32:0/58269737 shutdown_connections 2024-05-03T18:47:35.744 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:35.734+0000 7f3f94b64700 1 -- 172.21.15.32:0/58269737 wait complete. 2024-05-03T18:47:35.747 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:47:35 smithi045 ceph-mon[33299]: pgmap v366: 97 pgs: 97 active+clean; 631 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:47:35.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:47:35 smithi113 ceph-mon[33540]: pgmap v366: 97 pgs: 97 active+clean; 631 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:47:36.604 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid f500455e-097b-11ef-bc97-c7b262605968 -- ceph tell mds.c status 2024-05-03T18:47:36.861 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:47:37.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:47:37 smithi032 ceph-mon[29763]: pgmap v367: 97 pgs: 97 active+clean; 631 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:47:37.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:47:37 smithi045 ceph-mon[33299]: pgmap v367: 97 pgs: 97 active+clean; 631 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:47:37.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:47:37 smithi113 ceph-mon[33540]: pgmap v367: 97 pgs: 97 active+clean; 631 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:47:38.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.425+0000 7fc946b53700 1 -- 172.21.15.32:0/3822388995 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc940108850 msgr2=0x7fc9401103d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:38.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.425+0000 7fc946b53700 1 --2- 172.21.15.32:0/3822388995 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc940108850 0x7fc9401103d0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fc93800b0d0 tx=0x7fc93800b3e0 comp rx=0 tx=0).stop 2024-05-03T18:47:38.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.425+0000 7fc946b53700 1 -- 172.21.15.32:0/3822388995 shutdown_connections 2024-05-03T18:47:38.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.425+0000 7fc946b53700 1 --2- 172.21.15.32:0/3822388995 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc940108850 0x7fc9401103d0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:38.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.425+0000 7fc946b53700 1 --2- 172.21.15.32:0/3822388995 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc940103750 0x7fc940103b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:38.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.425+0000 7fc946b53700 1 --2- 172.21.15.32:0/3822388995 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc940104100 0x7fc940108120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:38.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.425+0000 7fc946b53700 1 -- 172.21.15.32:0/3822388995 >> 172.21.15.32:0/3822388995 conn(0x7fc94007a600 msgr2=0x7fc94007aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:47:38.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.425+0000 7fc946b53700 1 -- 172.21.15.32:0/3822388995 shutdown_connections 2024-05-03T18:47:38.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.425+0000 7fc946b53700 1 -- 172.21.15.32:0/3822388995 wait complete. 2024-05-03T18:47:38.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.426+0000 7fc946b53700 1 Processor -- start 2024-05-03T18:47:38.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.426+0000 7fc946b53700 1 -- start start 2024-05-03T18:47:38.429 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.426+0000 7fc946b53700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc940103750 0x7fc94019bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:38.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.426+0000 7fc946b53700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc940104100 0x7fc94019c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:38.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.426+0000 7fc946b53700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc940108850 0x7fc9401a0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:38.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.426+0000 7fc946b53700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fc940113710 con 0x7fc940108850 2024-05-03T18:47:38.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.426+0000 7fc946b53700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc940113590 con 0x7fc940103750 2024-05-03T18:47:38.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.426+0000 7fc946b53700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fc940113890 con 0x7fc940104100 2024-05-03T18:47:38.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.426+0000 7fc9450f0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc940108850 0x7fc9401a0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:38.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.426+0000 7fc9450f0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc940108850 0x7fc9401a0690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34426/0 (socket says 172.21.15.32:34426) 2024-05-03T18:47:38.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.426+0000 7fc9450f0700 1 -- 172.21.15.32:0/2216287096 learned_addr learned my addr 172.21.15.32:0/2216287096 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:47:38.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.426+0000 7fc9448ef700 1 --2- 172.21.15.32:0/2216287096 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc940103750 0x7fc94019bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:38.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.426+0000 7fc93ffff700 1 --2- 172.21.15.32:0/2216287096 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc940104100 0x7fc94019c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:38.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.426+0000 7fc9450f0700 1 -- 172.21.15.32:0/2216287096 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc940104100 msgr2=0x7fc94019c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:38.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.426+0000 7fc9450f0700 1 --2- 172.21.15.32:0/2216287096 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc940104100 0x7fc94019c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:38.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.426+0000 7fc9450f0700 1 -- 172.21.15.32:0/2216287096 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc940103750 msgr2=0x7fc94019bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:38.430 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.426+0000 7fc9450f0700 1 --2- 172.21.15.32:0/2216287096 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc940103750 0x7fc94019bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:38.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.426+0000 7fc9450f0700 1 -- 172.21.15.32:0/2216287096 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9401a0dd0 con 0x7fc940108850 2024-05-03T18:47:38.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.426+0000 7fc93ffff700 1 --2- 172.21.15.32:0/2216287096 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc940104100 0x7fc94019c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:47:38.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.426+0000 7fc9448ef700 1 --2- 172.21.15.32:0/2216287096 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc940103750 0x7fc94019bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T18:47:38.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.427+0000 7fc9450f0700 1 --2- 172.21.15.32:0/2216287096 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc940108850 0x7fc9401a0690 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fc938009fd0 tx=0x7fc938007ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:47:38.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.427+0000 7fc93dffb700 1 -- 172.21.15.32:0/2216287096 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc93801f070 con 0x7fc940108850 2024-05-03T18:47:38.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.427+0000 7fc93dffb700 1 -- 172.21.15.32:0/2216287096 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc938003c00 con 0x7fc940108850 2024-05-03T18:47:38.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.427+0000 7fc93dffb700 1 -- 172.21.15.32:0/2216287096 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc93801b8e0 con 0x7fc940108850 2024-05-03T18:47:38.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.427+0000 7fc946b53700 1 -- 172.21.15.32:0/2216287096 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9401a1060 con 0x7fc940108850 2024-05-03T18:47:38.431 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.427+0000 7fc946b53700 1 -- 172.21.15.32:0/2216287096 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9401a8940 con 0x7fc940108850 2024-05-03T18:47:38.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.429+0000 7fc93dffb700 1 -- 172.21.15.32:0/2216287096 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fc938022070 con 0x7fc940108850 2024-05-03T18:47:38.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.430+0000 7fc93dffb700 1 --2- 172.21.15.32:0/2216287096 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fc9280b2990 0x7fc9280b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:38.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.430+0000 7fc9448ef700 1 --2- 172.21.15.32:0/2216287096 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fc9280b2990 0x7fc9280b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:38.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.430+0000 7fc93dffb700 1 -- 172.21.15.32:0/2216287096 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8356+0+0 (secure 0 0 0) 0x7fc9380e35c0 con 0x7fc940108850 2024-05-03T18:47:38.432 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.431+0000 7fc9448ef700 1 --2- 172.21.15.32:0/2216287096 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fc9280b2990 0x7fc9280b4e50 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc9300099d0 tx=0x7fc930008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:47:38.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.432+0000 7fc9513e6b80 1 Processor -- start 2024-05-03T18:47:38.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.432+0000 7fc9513e6b80 1 -- start start 2024-05-03T18:47:38.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.432+0000 7fc9513e6b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ac810c0c10 0x55ac810c0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:38.434 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.432+0000 7fc9513e6b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ac810c1530 0x55ac810c4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:38.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.432+0000 7fc9513e6b80 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ac810c5230 0x55ac810c5690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:38.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.432+0000 7fc9513e6b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55ac81013810 con 0x55ac810c1530 2024-05-03T18:47:38.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.432+0000 7fc9513e6b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55ac80fb2770 con 0x55ac810c0c10 2024-05-03T18:47:38.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.432+0000 7fc93ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ac810c1530 0x55ac810c4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:38.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.432+0000 7fc9513e6b80 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55ac8100f7d0 con 0x55ac810c5230 2024-05-03T18:47:38.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.432+0000 7fc93ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ac810c1530 0x55ac810c4cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34430/0 (socket says 172.21.15.32:34430) 2024-05-03T18:47:38.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.432+0000 7fc93ffff700 1 -- 172.21.15.32:0/940984357 learned_addr learned my addr 172.21.15.32:0/940984357 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:47:38.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.433+0000 7fc9450f0700 1 --2- 172.21.15.32:0/940984357 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ac810c5230 0x55ac810c5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:38.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.433+0000 7fc9448ef700 1 --2- 172.21.15.32:0/940984357 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ac810c0c10 0x55ac810c0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:38.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.433+0000 7fc93ffff700 1 -- 172.21.15.32:0/940984357 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ac810c5230 msgr2=0x55ac810c5690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:47:38.435 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.433+0000 7fc93ffff700 1 --2- 172.21.15.32:0/940984357 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ac810c5230 0x55ac810c5690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:38.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.433+0000 7fc93ffff700 1 -- 172.21.15.32:0/940984357 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ac810c0c10 msgr2=0x55ac810c0ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:47:38.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.433+0000 7fc93ffff700 1 --2- 172.21.15.32:0/940984357 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ac810c0c10 0x55ac810c0ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:38.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.433+0000 7fc93ffff700 1 -- 172.21.15.32:0/940984357 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac80f60880 con 0x55ac810c1530 2024-05-03T18:47:38.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.433+0000 7fc9448ef700 1 --2- 172.21.15.32:0/940984357 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ac810c0c10 0x55ac810c0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T18:47:38.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.433+0000 7fc93ffff700 1 --2- 172.21.15.32:0/940984357 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ac810c1530 0x55ac810c4cf0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fc93400aa70 tx=0x7fc93400ad80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49bdb852e62b133b server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:47:38.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.433+0000 7fc926ffd700 1 -- 172.21.15.32:0/940984357 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc934015070 con 0x55ac810c1530 2024-05-03T18:47:38.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.433+0000 7fc926ffd700 1 -- 172.21.15.32:0/940984357 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc9340113f0 con 0x55ac810c1530 2024-05-03T18:47:38.436 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.433+0000 7fc926ffd700 1 -- 172.21.15.32:0/940984357 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9340185c0 con 0x55ac810c1530 2024-05-03T18:47:38.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.434+0000 7fc9513e6b80 1 -- 172.21.15.32:0/940984357 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ac810c1530 msgr2=0x55ac810c4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:38.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.434+0000 7fc9513e6b80 1 --2- 172.21.15.32:0/940984357 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ac810c1530 0x55ac810c4cf0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fc93400aa70 tx=0x7fc93400ad80 comp rx=0 tx=0).stop 2024-05-03T18:47:38.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.434+0000 7fc9513e6b80 1 -- 172.21.15.32:0/940984357 shutdown_connections 2024-05-03T18:47:38.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.434+0000 7fc9513e6b80 1 --2- 172.21.15.32:0/940984357 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ac810c5230 0x55ac810c5690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:38.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.434+0000 7fc9513e6b80 1 --2- 172.21.15.32:0/940984357 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ac810c0c10 0x55ac810c0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:38.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.434+0000 7fc9513e6b80 1 --2- 172.21.15.32:0/940984357 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ac810c1530 0x55ac810c4cf0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:38.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.434+0000 7fc9513e6b80 1 -- 172.21.15.32:0/940984357 >> 172.21.15.32:0/940984357 conn(0x55ac80fed0b0 msgr2=0x55ac810bb280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:47:38.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.434+0000 7fc9513e6b80 1 -- 172.21.15.32:0/940984357 shutdown_connections 2024-05-03T18:47:38.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.434+0000 7fc9513e6b80 1 -- 172.21.15.32:0/940984357 wait complete. 2024-05-03T18:47:38.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.434+0000 7fc9513e6b80 1 -- start start 2024-05-03T18:47:38.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.435+0000 7fc9513e6b80 1 Processor -- start 2024-05-03T18:47:38.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.435+0000 7fc9513e6b80 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ac810c0c10 0x55ac810cc5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:38.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.435+0000 7fc9513e6b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ac810c1530 0x55ac810c99d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:38.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.435+0000 7fc9513e6b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ac810c5230 0x55ac810c9f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:38.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.435+0000 7fc9513e6b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55ac810cd490 con 0x55ac810c1530 2024-05-03T18:47:38.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.435+0000 7fc9513e6b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55ac80f79be0 con 0x55ac810c5230 2024-05-03T18:47:38.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.435+0000 7fc9513e6b80 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55ac810cd610 con 0x55ac810c0c10 2024-05-03T18:47:38.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.435+0000 7fc9513e6b80 10 client.0 _ll_register_callbacks cb 0x55ac80fab8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:47:38.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.435+0000 7fc93ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ac810c1530 0x55ac810c99d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:38.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.435+0000 7fc93ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ac810c1530 0x55ac810c99d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34438/0 (socket says 172.21.15.32:34438) 2024-05-03T18:47:38.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.435+0000 7fc93ffff700 1 -- 172.21.15.32:0/3737181035 learned_addr learned my addr 172.21.15.32:0/3737181035 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:47:38.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.435+0000 7fc9450f0700 1 --2- 172.21.15.32:0/3737181035 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ac810c5230 0x55ac810c9f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:38.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.435+0000 7fc9448ef700 1 --2- 172.21.15.32:0/3737181035 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ac810c0c10 0x55ac810cc5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:38.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.435+0000 7fc93ffff700 1 -- 172.21.15.32:0/3737181035 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ac810c0c10 msgr2=0x55ac810cc5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:47:38.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.435+0000 7fc93ffff700 1 --2- 172.21.15.32:0/3737181035 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ac810c0c10 0x55ac810cc5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:38.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.435+0000 7fc93ffff700 1 -- 172.21.15.32:0/3737181035 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ac810c5230 msgr2=0x55ac810c9f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:47:38.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.435+0000 7fc93ffff700 1 --2- 172.21.15.32:0/3737181035 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ac810c5230 0x55ac810c9f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:38.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.435+0000 7fc93ffff700 1 -- 172.21.15.32:0/3737181035 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac80faf640 con 0x55ac810c1530 2024-05-03T18:47:38.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.436+0000 7fc93ffff700 1 --2- 172.21.15.32:0/3737181035 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ac810c1530 0x55ac810c99d0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fc934011b50 tx=0x7fc934011b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5953d7a6b9982d80 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:47:38.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.436+0000 7fc91ffff700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-05-03T18:47:38.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.436+0000 7fc91ffff700 1 -- 172.21.15.32:0/3737181035 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc934015040 con 0x55ac810c1530 2024-05-03T18:47:38.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.436+0000 7fc91ffff700 1 -- 172.21.15.32:0/3737181035 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8356+0+0 (secure 0 0 0) 0x7fc93400cca0 con 0x55ac810c1530 2024-05-03T18:47:38.442 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.436+0000 7fc9513e6b80 1 -- 172.21.15.32:0/3737181035 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc9340113f0 con 0x55ac810c1530 2024-05-03T18:47:38.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.436+0000 7fc91ffff700 1 -- 172.21.15.32:0/3737181035 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc93400cea0 con 0x55ac810c1530 2024-05-03T18:47:38.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.436+0000 7fc9513e6b80 10 client.15108 fetch_fsmap learned FSMap version 25 2024-05-03T18:47:38.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.436+0000 7fc9513e6b80 1 -- 172.21.15.32:0/3737181035 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ac810d7c10 con 0x55ac810c1530 2024-05-03T18:47:38.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.436+0000 7fc91ffff700 1 -- 172.21.15.32:0/3737181035 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc9340113f0 con 0x55ac810c1530 2024-05-03T18:47:38.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.436+0000 7fc91ffff700 1 -- 172.21.15.32:0/3737181035 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fc9340253f0 con 0x55ac810c1530 2024-05-03T18:47:38.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.436+0000 7fc9513e6b80 10 client.15108 fetch_fsmap finished waiting for FSMap version 25 2024-05-03T18:47:38.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.436+0000 7fc9513e6b80 10 client.15108 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-03T18:47:38.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.436+0000 7fc9513e6b80 20 client.15108 populate_metadata read hostname 'smithi032' 2024-05-03T18:47:38.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.437+0000 7fc9513e6b80 1 --2- 172.21.15.32:0/3737181035 >> [v2:172.21.15.113:6836/3677284460,v1:172.21.15.113:6837/3677284460] conn(0x55ac810d9e90 0x55ac810db300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:38.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.437+0000 7fc9513e6b80 4 client.15108 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-03T18:47:38.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.437+0000 7fc9513e6b80 1 -- 172.21.15.32:0/3737181035 --> [v2:172.21.15.113:6836/3677284460,v1:172.21.15.113:6837/3677284460] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ac810211f0 con 0x55ac810d9e90 2024-05-03T18:47:38.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.437+0000 7fc9448ef700 1 --2- 172.21.15.32:0/3737181035 >> [v2:172.21.15.113:6836/3677284460,v1:172.21.15.113:6837/3677284460] conn(0x55ac810d9e90 0x55ac810db300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:38.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.437+0000 7fc9448ef700 1 --2- 172.21.15.32:0/3737181035 >> [v2:172.21.15.113:6836/3677284460,v1:172.21.15.113:6837/3677284460] conn(0x55ac810d9e90 0x55ac810db300 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bf6f1fd7b36d8dfe server_cookie=174771f549f7eaff in_seq=0 out_seq=0 2024-05-03T18:47:38.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.437+0000 7fc91ffff700 10 client.15108 ms_handle_connect on v2:172.21.15.113:6836/3677284460 2024-05-03T18:47:38.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.438+0000 7fc91ffff700 1 -- 172.21.15.32:0/3737181035 <== mds.1 v2:172.21.15.113:6836/3677284460 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ac810211f0 con 0x55ac810d9e90 2024-05-03T18:47:38.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.438+0000 7fc91ffff700 10 client.15108 handle_command_reply: tid=0 2024-05-03T18:47:38.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.438+0000 7fc9513e6b80 1 client.15108 shutdown 2024-05-03T18:47:38.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.438+0000 7fc9513e6b80 1 -- 172.21.15.32:0/3737181035 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ac810c1530 msgr2=0x55ac810c99d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:38.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.438+0000 7fc9513e6b80 1 --2- 172.21.15.32:0/3737181035 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ac810c1530 0x55ac810c99d0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fc934011b50 tx=0x7fc934011b80 comp rx=0 tx=0).stop 2024-05-03T18:47:38.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.438+0000 7fc9513e6b80 1 -- 172.21.15.32:0/3737181035 shutdown_connections 2024-05-03T18:47:38.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.438+0000 7fc9513e6b80 1 --2- 172.21.15.32:0/3737181035 >> [v2:172.21.15.113:6836/3677284460,v1:172.21.15.113:6837/3677284460] conn(0x55ac810d9e90 0x55ac810db300 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:38.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.438+0000 7fc9513e6b80 1 --2- 172.21.15.32:0/3737181035 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ac810c5230 0x55ac810c9f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:38.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.438+0000 7fc9513e6b80 1 --2- 172.21.15.32:0/3737181035 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ac810c0c10 0x55ac810cc5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:38.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.438+0000 7fc91ffff700 0 client.15108 ms_handle_reset on v2:172.21.15.113:6836/3677284460 2024-05-03T18:47:38.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.438+0000 7fc9513e6b80 1 --2- 172.21.15.32:0/3737181035 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ac810c1530 0x55ac810c99d0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:38.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.438+0000 7fc9513e6b80 1 -- 172.21.15.32:0/3737181035 >> 172.21.15.32:0/3737181035 conn(0x55ac80fed0b0 msgr2=0x55ac810c66e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:47:38.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.438+0000 7fc9513e6b80 1 -- 172.21.15.32:0/3737181035 shutdown_connections 2024-05-03T18:47:38.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.438+0000 7fc9513e6b80 1 -- 172.21.15.32:0/3737181035 wait complete. 2024-05-03T18:47:38.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.438+0000 7fc9513e6b80 20 client.15108 trim_cache size 0 max 16384 2024-05-03T18:47:38.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.459+0000 7fc9513e6b80 1 Processor -- start 2024-05-03T18:47:38.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.459+0000 7fc9513e6b80 1 -- start start 2024-05-03T18:47:38.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.459+0000 7fc9513e6b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ac810c0c10 0x55ac810d97c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:38.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.459+0000 7fc9513e6b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ac810c1530 0x55ac810bb070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:38.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.459+0000 7fc9513e6b80 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ac810c5230 0x55ac810c6db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:38.461 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.459+0000 7fc9513e6b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55ac810d9640 con 0x55ac810c1530 2024-05-03T18:47:38.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.459+0000 7fc9513e6b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55ac810d9d10 con 0x55ac810c0c10 2024-05-03T18:47:38.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.459+0000 7fc9513e6b80 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55ac810da3e0 con 0x55ac810c5230 2024-05-03T18:47:38.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.460+0000 7fc93ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ac810c1530 0x55ac810bb070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:38.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.460+0000 7fc93ffff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ac810c1530 0x55ac810bb070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34448/0 (socket says 172.21.15.32:34448) 2024-05-03T18:47:38.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.460+0000 7fc93ffff700 1 -- 172.21.15.32:0/3689941514 learned_addr learned my addr 172.21.15.32:0/3689941514 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:47:38.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.460+0000 7fc9450f0700 1 --2- 172.21.15.32:0/3689941514 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ac810c5230 0x55ac810c6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:38.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.460+0000 7fc93ffff700 1 -- 172.21.15.32:0/3689941514 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ac810c5230 msgr2=0x55ac810c6db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:47:38.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.460+0000 7fc9448ef700 1 --2- 172.21.15.32:0/3689941514 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ac810c0c10 0x55ac810d97c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:38.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.460+0000 7fc93ffff700 1 --2- 172.21.15.32:0/3689941514 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ac810c5230 0x55ac810c6db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:38.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.460+0000 7fc93ffff700 1 -- 172.21.15.32:0/3689941514 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ac810c0c10 msgr2=0x55ac810d97c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:47:38.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.460+0000 7fc93ffff700 1 --2- 172.21.15.32:0/3689941514 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ac810c0c10 0x55ac810d97c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:38.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.460+0000 7fc93ffff700 1 -- 172.21.15.32:0/3689941514 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac810ca290 con 0x55ac810c1530 2024-05-03T18:47:38.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.460+0000 7fc93ffff700 1 --2- 172.21.15.32:0/3689941514 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ac810c1530 0x55ac810bb070 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fc9340048a0 tx=0x7fc934006cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4096168af8ecac61 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:47:38.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.460+0000 7fc9267fc700 1 -- 172.21.15.32:0/3689941514 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc934032020 con 0x55ac810c1530 2024-05-03T18:47:38.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.460+0000 7fc9267fc700 1 -- 172.21.15.32:0/3689941514 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc9340113f0 con 0x55ac810c1530 2024-05-03T18:47:38.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.460+0000 7fc9267fc700 1 -- 172.21.15.32:0/3689941514 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc934018810 con 0x55ac810c1530 2024-05-03T18:47:38.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.461+0000 7fc9513e6b80 1 -- 172.21.15.32:0/3689941514 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ac810c1530 msgr2=0x55ac810bb070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:38.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.461+0000 7fc9513e6b80 1 --2- 172.21.15.32:0/3689941514 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ac810c1530 0x55ac810bb070 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fc9340048a0 tx=0x7fc934006cb0 comp rx=0 tx=0).stop 2024-05-03T18:47:38.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.461+0000 7fc9513e6b80 1 -- 172.21.15.32:0/3689941514 shutdown_connections 2024-05-03T18:47:38.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.461+0000 7fc9513e6b80 1 --2- 172.21.15.32:0/3689941514 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ac810c5230 0x55ac810c6db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:38.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.461+0000 7fc9513e6b80 1 --2- 172.21.15.32:0/3689941514 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ac810c0c10 0x55ac810d97c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:38.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.461+0000 7fc9513e6b80 1 --2- 172.21.15.32:0/3689941514 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ac810c1530 0x55ac810bb070 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:38.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.461+0000 7fc9513e6b80 1 -- 172.21.15.32:0/3689941514 >> 172.21.15.32:0/3689941514 conn(0x55ac80fed0b0 msgr2=0x55ac810c02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:47:38.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.461+0000 7fc9513e6b80 1 -- 172.21.15.32:0/3689941514 shutdown_connections 2024-05-03T18:47:38.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.461+0000 7fc9513e6b80 1 -- 172.21.15.32:0/3689941514 wait complete. 2024-05-03T18:47:38.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.461+0000 7fc9513e6b80 1 -- start start 2024-05-03T18:47:38.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.461+0000 7fc9513e6b80 1 Processor -- start 2024-05-03T18:47:38.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.462+0000 7fc9513e6b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ac810c0c10 0x55ac810e4780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:38.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.462+0000 7fc9513e6b80 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ac810c1530 0x55ac810e4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:38.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.462+0000 7fc9513e6b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ac810c5230 0x55ac810dd720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:38.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.462+0000 7fc9513e6b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x55ac810db180 con 0x55ac810c5230 2024-05-03T18:47:38.467 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.462+0000 7fc9513e6b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55ac810daab0 con 0x55ac810c0c10 2024-05-03T18:47:38.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.462+0000 7fc9513e6b80 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55ac810db850 con 0x55ac810c1530 2024-05-03T18:47:38.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.462+0000 7fc9513e6b80 10 client.0 _ll_register_callbacks cb 0x55ac80fab8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:47:38.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.462+0000 7fc9450f0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ac810c5230 0x55ac810dd720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:38.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.462+0000 7fc9450f0700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ac810c5230 0x55ac810dd720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34450/0 (socket says 172.21.15.32:34450) 2024-05-03T18:47:38.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.462+0000 7fc9450f0700 1 -- 172.21.15.32:0/60033817 learned_addr learned my addr 172.21.15.32:0/60033817 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:47:38.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.462+0000 7fc9448ef700 1 --2- 172.21.15.32:0/60033817 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ac810c0c10 0x55ac810e4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:38.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.462+0000 7fc93ffff700 1 --2- 172.21.15.32:0/60033817 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ac810c1530 0x55ac810e4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:38.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.462+0000 7fc9450f0700 1 -- 172.21.15.32:0/60033817 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ac810c1530 msgr2=0x55ac810e4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:47:38.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.462+0000 7fc9450f0700 1 --2- 172.21.15.32:0/60033817 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ac810c1530 0x55ac810e4cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:38.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.462+0000 7fc9450f0700 1 -- 172.21.15.32:0/60033817 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ac810c0c10 msgr2=0x55ac810e4780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:47:38.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.462+0000 7fc9450f0700 1 --2- 172.21.15.32:0/60033817 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ac810c0c10 0x55ac810e4780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:38.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.462+0000 7fc9450f0700 1 -- 172.21.15.32:0/60033817 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac810ddf60 con 0x55ac810c5230 2024-05-03T18:47:38.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.462+0000 7fc9448ef700 1 --2- 172.21.15.32:0/60033817 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ac810c0c10 0x55ac810e4780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:47:38.468 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.462+0000 7fc93ffff700 1 --2- 172.21.15.32:0/60033817 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ac810c1530 0x55ac810e4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:47:38.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.462+0000 7fc9450f0700 1 --2- 172.21.15.32:0/60033817 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ac810c5230 0x55ac810dd720 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fc9380e0f20 tx=0x7fc9380060a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f04e6c0897f75c96 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:47:38.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.462+0000 7fc91ffff700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-05-03T18:47:38.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.462+0000 7fc91ffff700 1 -- 172.21.15.32:0/60033817 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc938029040 con 0x55ac810c5230 2024-05-03T18:47:38.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.462+0000 7fc91ffff700 1 -- 172.21.15.32:0/60033817 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8356+0+0 (secure 0 0 0) 0x7fc9380aacf0 con 0x55ac810c5230 2024-05-03T18:47:38.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.462+0000 7fc9513e6b80 1 -- 172.21.15.32:0/60033817 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc9340113f0 con 0x55ac810c5230 2024-05-03T18:47:38.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.463+0000 7fc91ffff700 1 -- 172.21.15.32:0/60033817 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc9380aaa90 con 0x55ac810c5230 2024-05-03T18:47:38.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.463+0000 7fc9513e6b80 10 client.15114 fetch_fsmap learned FSMap version 25 2024-05-03T18:47:38.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.463+0000 7fc9513e6b80 1 -- 172.21.15.32:0/60033817 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ac81145dc0 con 0x55ac810c5230 2024-05-03T18:47:38.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.463+0000 7fc91ffff700 1 -- 172.21.15.32:0/60033817 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc9340113f0 con 0x55ac810c5230 2024-05-03T18:47:38.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.463+0000 7fc91ffff700 1 -- 172.21.15.32:0/60033817 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fc9380c6030 con 0x55ac810c5230 2024-05-03T18:47:38.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.463+0000 7fc9513e6b80 10 client.15114 fetch_fsmap finished waiting for FSMap version 25 2024-05-03T18:47:38.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.463+0000 7fc9513e6b80 10 client.15114 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-03T18:47:38.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.463+0000 7fc9513e6b80 20 client.15114 populate_metadata read hostname 'smithi032' 2024-05-03T18:47:38.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.463+0000 7fc9513e6b80 1 --2- 172.21.15.32:0/60033817 >> [v2:172.21.15.113:6836/3677284460,v1:172.21.15.113:6837/3677284460] conn(0x55ac8114aa80 0x55ac81149790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:38.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.463+0000 7fc9513e6b80 4 client.15114 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-03T18:47:38.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.463+0000 7fc9513e6b80 1 -- 172.21.15.32:0/60033817 --> [v2:172.21.15.113:6836/3677284460,v1:172.21.15.113:6837/3677284460] -- command(tid 0: {"prefix": "status"}) -- 0x55ac8101b290 con 0x55ac8114aa80 2024-05-03T18:47:38.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.463+0000 7fc9448ef700 1 --2- 172.21.15.32:0/60033817 >> [v2:172.21.15.113:6836/3677284460,v1:172.21.15.113:6837/3677284460] conn(0x55ac8114aa80 0x55ac81149790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:38.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.464+0000 7fc9448ef700 1 --2- 172.21.15.32:0/60033817 >> [v2:172.21.15.113:6836/3677284460,v1:172.21.15.113:6837/3677284460] conn(0x55ac8114aa80 0x55ac81149790 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8583dcbf1215e0b5 server_cookie=491de92770f9430 in_seq=0 out_seq=0 2024-05-03T18:47:38.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.464+0000 7fc91ffff700 10 client.15114 ms_handle_connect on v2:172.21.15.113:6836/3677284460 2024-05-03T18:47:38.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.464+0000 7fc91ffff700 1 -- 172.21.15.32:0/60033817 <== mds.1 v2:172.21.15.113:6836/3677284460 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55ac8101b290 con 0x55ac8114aa80 2024-05-03T18:47:38.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.464+0000 7fc91ffff700 10 client.15114 handle_command_reply: tid=0 2024-05-03T18:47:38.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.464+0000 7fc9513e6b80 1 client.15114 shutdown 2024-05-03T18:47:38.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.464+0000 7fc9513e6b80 1 -- 172.21.15.32:0/60033817 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ac810c5230 msgr2=0x55ac810dd720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:38.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.464+0000 7fc9513e6b80 1 --2- 172.21.15.32:0/60033817 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ac810c5230 0x55ac810dd720 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fc9380e0f20 tx=0x7fc9380060a0 comp rx=0 tx=0).stop 2024-05-03T18:47:38.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.464+0000 7fc9513e6b80 1 -- 172.21.15.32:0/60033817 shutdown_connections 2024-05-03T18:47:38.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.464+0000 7fc9513e6b80 1 --2- 172.21.15.32:0/60033817 >> [v2:172.21.15.113:6836/3677284460,v1:172.21.15.113:6837/3677284460] conn(0x55ac8114aa80 0x55ac81149790 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:38.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.464+0000 7fc9513e6b80 1 --2- 172.21.15.32:0/60033817 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55ac810c5230 0x55ac810dd720 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:38.470 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.464+0000 7fc9513e6b80 1 --2- 172.21.15.32:0/60033817 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ac810c0c10 0x55ac810e4780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:38.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.464+0000 7fc9513e6b80 1 --2- 172.21.15.32:0/60033817 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ac810c1530 0x55ac810e4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:38.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.464+0000 7fc9513e6b80 1 -- 172.21.15.32:0/60033817 >> 172.21.15.32:0/60033817 conn(0x55ac80fed0b0 msgr2=0x55ac810caf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:47:38.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.464+0000 7fc91ffff700 0 client.15114 ms_handle_reset on v2:172.21.15.113:6836/3677284460 2024-05-03T18:47:38.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.464+0000 7fc9513e6b80 1 -- 172.21.15.32:0/60033817 shutdown_connections 2024-05-03T18:47:38.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.464+0000 7fc9513e6b80 1 -- 172.21.15.32:0/60033817 wait complete. 2024-05-03T18:47:38.471 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.464+0000 7fc9513e6b80 20 client.15114 trim_cache size 0 max 16384 2024-05-03T18:47:38.471 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-05-03T18:47:38.471 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_fsid": "f500455e-097b-11ef-bc97-c7b262605968", 2024-05-03T18:47:38.471 INFO:teuthology.orchestra.run.smithi032.stdout: "whoami": 1, 2024-05-03T18:47:38.471 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 24469, 2024-05-03T18:47:38.471 INFO:teuthology.orchestra.run.smithi032.stdout: "want_state": "up:active", 2024-05-03T18:47:38.472 INFO:teuthology.orchestra.run.smithi032.stdout: "state": "up:active", 2024-05-03T18:47:38.472 INFO:teuthology.orchestra.run.smithi032.stdout: "fs_name": "cephfs", 2024-05-03T18:47:38.472 INFO:teuthology.orchestra.run.smithi032.stdout: "rank_uptime": 155.99783453500001, 2024-05-03T18:47:38.472 INFO:teuthology.orchestra.run.smithi032.stdout: "mdsmap_epoch": 25, 2024-05-03T18:47:38.472 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch": 82, 2024-05-03T18:47:38.472 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch_barrier": 78, 2024-05-03T18:47:38.472 INFO:teuthology.orchestra.run.smithi032.stdout: "uptime": 196.15444728400001 2024-05-03T18:47:38.472 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-05-03T18:47:38.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.465+0000 7fc946b53700 1 -- 172.21.15.32:0/2216287096 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fc9280b2990 msgr2=0x7fc9280b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:38.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.465+0000 7fc946b53700 1 --2- 172.21.15.32:0/2216287096 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fc9280b2990 0x7fc9280b4e50 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc9300099d0 tx=0x7fc930008040 comp rx=0 tx=0).stop 2024-05-03T18:47:38.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.465+0000 7fc946b53700 1 -- 172.21.15.32:0/2216287096 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc940108850 msgr2=0x7fc9401a0690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:38.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.465+0000 7fc946b53700 1 --2- 172.21.15.32:0/2216287096 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc940108850 0x7fc9401a0690 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fc938009fd0 tx=0x7fc938007ad0 comp rx=0 tx=0).stop 2024-05-03T18:47:38.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.465+0000 7fc946b53700 1 -- 172.21.15.32:0/2216287096 shutdown_connections 2024-05-03T18:47:38.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.465+0000 7fc946b53700 1 --2- 172.21.15.32:0/2216287096 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fc9280b2990 0x7fc9280b4e50 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:38.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.465+0000 7fc946b53700 1 --2- 172.21.15.32:0/2216287096 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc940108850 0x7fc9401a0690 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:38.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.465+0000 7fc946b53700 1 --2- 172.21.15.32:0/2216287096 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc940103750 0x7fc94019bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:38.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.465+0000 7fc946b53700 1 --2- 172.21.15.32:0/2216287096 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc940104100 0x7fc94019c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:38.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.465+0000 7fc946b53700 1 -- 172.21.15.32:0/2216287096 >> 172.21.15.32:0/2216287096 conn(0x7fc94007a600 msgr2=0x7fc9401019d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:47:38.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.465+0000 7fc946b53700 1 -- 172.21.15.32:0/2216287096 shutdown_connections 2024-05-03T18:47:38.473 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:38.465+0000 7fc946b53700 1 -- 172.21.15.32:0/2216287096 wait complete. 2024-05-03T18:47:39.316 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid f500455e-097b-11ef-bc97-c7b262605968 -- ceph tell mds.i status 2024-05-03T18:47:39.565 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:47:39.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:47:39 smithi032 ceph-mon[29763]: pgmap v368: 97 pgs: 97 active+clean; 631 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:47:39.747 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:47:39 smithi045 ceph-mon[33299]: pgmap v368: 97 pgs: 97 active+clean; 631 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:47:39.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:47:39 smithi113 ceph-mon[33540]: pgmap v368: 97 pgs: 97 active+clean; 631 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:47:41.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.057+0000 7fdc6e47d700 1 -- 172.21.15.32:0/1128591738 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc6806bdd0 msgr2=0x7fdc6806c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:41.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.057+0000 7fdc6e47d700 1 --2- 172.21.15.32:0/1128591738 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc6806bdd0 0x7fdc6806c1b0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fdc580099c0 tx=0x7fdc58009cd0 comp rx=0 tx=0).stop 2024-05-03T18:47:41.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.057+0000 7fdc6e47d700 1 -- 172.21.15.32:0/1128591738 shutdown_connections 2024-05-03T18:47:41.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.057+0000 7fdc6e47d700 1 --2- 172.21.15.32:0/1128591738 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdc68108870 0x7fdc681103f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:41.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.057+0000 7fdc6e47d700 1 --2- 172.21.15.32:0/1128591738 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc6806bdd0 0x7fdc6806c1b0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:41.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.057+0000 7fdc6e47d700 1 --2- 172.21.15.32:0/1128591738 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc6806c780 0x7fdc68108140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:41.059 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.057+0000 7fdc6e47d700 1 -- 172.21.15.32:0/1128591738 >> 172.21.15.32:0/1128591738 conn(0x7fdc68079350 msgr2=0x7fdc68079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:47:41.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.057+0000 7fdc6e47d700 1 -- 172.21.15.32:0/1128591738 shutdown_connections 2024-05-03T18:47:41.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.057+0000 7fdc6e47d700 1 -- 172.21.15.32:0/1128591738 wait complete. 2024-05-03T18:47:41.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.058+0000 7fdc6e47d700 1 Processor -- start 2024-05-03T18:47:41.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.058+0000 7fdc6e47d700 1 -- start start 2024-05-03T18:47:41.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.058+0000 7fdc6e47d700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdc6806bdd0 0x7fdc6819f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:41.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.058+0000 7fdc6e47d700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc6806c780 0x7fdc6819fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:41.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.058+0000 7fdc6e47d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc68108870 0x7fdc68199960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:41.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.058+0000 7fdc6e47d700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fdc68113660 con 0x7fdc68108870 2024-05-03T18:47:41.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.058+0000 7fdc6e47d700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fdc681134e0 con 0x7fdc6806c780 2024-05-03T18:47:41.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.058+0000 7fdc6e47d700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fdc681137e0 con 0x7fdc6806bdd0 2024-05-03T18:47:41.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.058+0000 7fdc6ca1a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc68108870 0x7fdc68199960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:41.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.058+0000 7fdc6ca1a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc68108870 0x7fdc68199960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34468/0 (socket says 172.21.15.32:34468) 2024-05-03T18:47:41.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.058+0000 7fdc6ca1a700 1 -- 172.21.15.32:0/4212755952 learned_addr learned my addr 172.21.15.32:0/4212755952 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:47:41.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.058+0000 7fdc677fe700 1 --2- 172.21.15.32:0/4212755952 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc6806c780 0x7fdc6819fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:41.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.058+0000 7fdc6ca1a700 1 -- 172.21.15.32:0/4212755952 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdc6806bdd0 msgr2=0x7fdc6819f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:41.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.058+0000 7fdc67fff700 1 --2- 172.21.15.32:0/4212755952 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdc6806bdd0 0x7fdc6819f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:41.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.059+0000 7fdc6ca1a700 1 --2- 172.21.15.32:0/4212755952 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdc6806bdd0 0x7fdc6819f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:41.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.059+0000 7fdc6ca1a700 1 -- 172.21.15.32:0/4212755952 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc6806c780 msgr2=0x7fdc6819fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:41.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.059+0000 7fdc6ca1a700 1 --2- 172.21.15.32:0/4212755952 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc6806c780 0x7fdc6819fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:41.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.059+0000 7fdc6ca1a700 1 -- 172.21.15.32:0/4212755952 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc6819a0c0 con 0x7fdc68108870 2024-05-03T18:47:41.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.059+0000 7fdc677fe700 1 --2- 172.21.15.32:0/4212755952 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc6806c780 0x7fdc6819fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T18:47:41.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.059+0000 7fdc67fff700 1 --2- 172.21.15.32:0/4212755952 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdc6806bdd0 0x7fdc6819f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:47:41.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.059+0000 7fdc6ca1a700 1 --2- 172.21.15.32:0/4212755952 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc68108870 0x7fdc68199960 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fdc60007ee0 tx=0x7fdc6000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:47:41.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.059+0000 7fdc657fa700 1 -- 172.21.15.32:0/4212755952 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc6000dbd0 con 0x7fdc68108870 2024-05-03T18:47:41.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.059+0000 7fdc657fa700 1 -- 172.21.15.32:0/4212755952 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc6000dd70 con 0x7fdc68108870 2024-05-03T18:47:41.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.059+0000 7fdc6e47d700 1 -- 172.21.15.32:0/4212755952 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc6819a3b0 con 0x7fdc68108870 2024-05-03T18:47:41.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.059+0000 7fdc657fa700 1 -- 172.21.15.32:0/4212755952 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc60005990 con 0x7fdc68108870 2024-05-03T18:47:41.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.059+0000 7fdc6e47d700 1 -- 172.21.15.32:0/4212755952 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc681a6620 con 0x7fdc68108870 2024-05-03T18:47:41.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.061+0000 7fdc657fa700 1 -- 172.21.15.32:0/4212755952 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fdc6001a020 con 0x7fdc68108870 2024-05-03T18:47:41.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.062+0000 7fdc657fa700 1 --2- 172.21.15.32:0/4212755952 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fdc500b2910 0x7fdc500b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:41.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.062+0000 7fdc67fff700 1 --2- 172.21.15.32:0/4212755952 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fdc500b2910 0x7fdc500b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:41.076 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.062+0000 7fdc657fa700 1 -- 172.21.15.32:0/4212755952 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8356+0+0 (secure 0 0 0) 0x7fdc600e1c70 con 0x7fdc68108870 2024-05-03T18:47:41.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.063+0000 7fdc67fff700 1 --2- 172.21.15.32:0/4212755952 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fdc500b2910 0x7fdc500b4dd0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fdc58003800 tx=0x7fdc58005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:47:41.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.063+0000 7fdc78d10b80 1 Processor -- start 2024-05-03T18:47:41.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.063+0000 7fdc78d10b80 1 -- start start 2024-05-03T18:47:41.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.063+0000 7fdc78d10b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5623119bd740 0x5623119bdb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:41.077 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.063+0000 7fdc78d10b80 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5623119be060 0x5623119b8f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:41.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.063+0000 7fdc78d10b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5623119b9480 0x5623119b98c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:41.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.063+0000 7fdc78d10b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x56231190b8f0 con 0x5623119b9480 2024-05-03T18:47:41.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.063+0000 7fdc78d10b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x562311870d50 con 0x5623119bd740 2024-05-03T18:47:41.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.063+0000 7fdc78d10b80 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5623119078b0 con 0x5623119be060 2024-05-03T18:47:41.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.063+0000 7fdc6ca1a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5623119b9480 0x5623119b98c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:41.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.063+0000 7fdc6ca1a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5623119b9480 0x5623119b98c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34478/0 (socket says 172.21.15.32:34478) 2024-05-03T18:47:41.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.063+0000 7fdc6ca1a700 1 -- 172.21.15.32:0/4126849068 learned_addr learned my addr 172.21.15.32:0/4126849068 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:47:41.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.063+0000 7fdc67fff700 1 --2- 172.21.15.32:0/4126849068 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5623119bd740 0x5623119bdb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:41.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.063+0000 7fdc677fe700 1 --2- 172.21.15.32:0/4126849068 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5623119be060 0x5623119b8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:41.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.063+0000 7fdc6ca1a700 1 -- 172.21.15.32:0/4126849068 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5623119be060 msgr2=0x5623119b8f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:47:41.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.063+0000 7fdc6ca1a700 1 --2- 172.21.15.32:0/4126849068 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5623119be060 0x5623119b8f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:41.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.063+0000 7fdc6ca1a700 1 -- 172.21.15.32:0/4126849068 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5623119bd740 msgr2=0x5623119bdb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:47:41.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.063+0000 7fdc6ca1a700 1 --2- 172.21.15.32:0/4126849068 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5623119bd740 0x5623119bdb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:41.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.063+0000 7fdc6ca1a700 1 -- 172.21.15.32:0/4126849068 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562311858880 con 0x5623119b9480 2024-05-03T18:47:41.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.063+0000 7fdc677fe700 1 --2- 172.21.15.32:0/4126849068 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5623119be060 0x5623119b8f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:47:41.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.064+0000 7fdc6ca1a700 1 --2- 172.21.15.32:0/4126849068 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5623119b9480 0x5623119b98c0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fdc6000dfd0 tx=0x7fdc600a88a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=781643c99b21782b server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:47:41.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.064+0000 7fdc4e7fc700 1 -- 172.21.15.32:0/4126849068 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc600a90b0 con 0x5623119b9480 2024-05-03T18:47:41.080 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.064+0000 7fdc4e7fc700 1 -- 172.21.15.32:0/4126849068 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc600ad460 con 0x5623119b9480 2024-05-03T18:47:41.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.064+0000 7fdc4e7fc700 1 -- 172.21.15.32:0/4126849068 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc600a9550 con 0x5623119b9480 2024-05-03T18:47:41.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.064+0000 7fdc78d10b80 1 -- 172.21.15.32:0/4126849068 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5623119b9480 msgr2=0x5623119b98c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:41.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.064+0000 7fdc78d10b80 1 --2- 172.21.15.32:0/4126849068 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5623119b9480 0x5623119b98c0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fdc6000dfd0 tx=0x7fdc600a88a0 comp rx=0 tx=0).stop 2024-05-03T18:47:41.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.065+0000 7fdc78d10b80 1 -- 172.21.15.32:0/4126849068 shutdown_connections 2024-05-03T18:47:41.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.065+0000 7fdc78d10b80 1 --2- 172.21.15.32:0/4126849068 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5623119b9480 0x5623119b98c0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:41.081 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.065+0000 7fdc78d10b80 1 --2- 172.21.15.32:0/4126849068 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5623119bd740 0x5623119bdb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:41.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.065+0000 7fdc78d10b80 1 --2- 172.21.15.32:0/4126849068 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5623119be060 0x5623119b8f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:41.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.065+0000 7fdc78d10b80 1 -- 172.21.15.32:0/4126849068 >> 172.21.15.32:0/4126849068 conn(0x5623118e5b20 msgr2=0x5623119b39a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:47:41.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.065+0000 7fdc78d10b80 1 -- 172.21.15.32:0/4126849068 shutdown_connections 2024-05-03T18:47:41.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.065+0000 7fdc78d10b80 1 -- 172.21.15.32:0/4126849068 wait complete. 2024-05-03T18:47:41.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.065+0000 7fdc78d10b80 1 -- start start 2024-05-03T18:47:41.082 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.065+0000 7fdc78d10b80 1 Processor -- start 2024-05-03T18:47:41.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.065+0000 7fdc78d10b80 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5623119b9480 0x5623119c4d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:41.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.065+0000 7fdc78d10b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5623119bd740 0x5623119c2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:41.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.065+0000 7fdc78d10b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5623119be060 0x5623119c2650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:41.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.066+0000 7fdc78d10b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5623119c5ba0 con 0x5623119be060 2024-05-03T18:47:41.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.066+0000 7fdc78d10b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5623118c4e80 con 0x5623119bd740 2024-05-03T18:47:41.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.066+0000 7fdc78d10b80 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5623119c5d20 con 0x5623119b9480 2024-05-03T18:47:41.083 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.066+0000 7fdc78d10b80 10 client.0 _ll_register_callbacks cb 0x56231189f560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:47:41.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.066+0000 7fdc6ca1a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5623119be060 0x5623119c2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:41.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.066+0000 7fdc6ca1a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5623119be060 0x5623119c2650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34482/0 (socket says 172.21.15.32:34482) 2024-05-03T18:47:41.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.066+0000 7fdc6ca1a700 1 -- 172.21.15.32:0/2882637643 learned_addr learned my addr 172.21.15.32:0/2882637643 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:47:41.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.066+0000 7fdc67fff700 1 --2- 172.21.15.32:0/2882637643 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5623119b9480 0x5623119c4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:41.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.066+0000 7fdc677fe700 1 --2- 172.21.15.32:0/2882637643 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5623119bd740 0x5623119c2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:41.084 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.066+0000 7fdc6ca1a700 1 -- 172.21.15.32:0/2882637643 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5623119b9480 msgr2=0x5623119c4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:47:41.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.066+0000 7fdc6ca1a700 1 --2- 172.21.15.32:0/2882637643 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5623119b9480 0x5623119c4d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:41.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.066+0000 7fdc6ca1a700 1 -- 172.21.15.32:0/2882637643 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5623119bd740 msgr2=0x5623119c2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:47:41.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.066+0000 7fdc6ca1a700 1 --2- 172.21.15.32:0/2882637643 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5623119bd740 0x5623119c2110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:41.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.066+0000 7fdc6ca1a700 1 -- 172.21.15.32:0/2882637643 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623118c2710 con 0x5623119be060 2024-05-03T18:47:41.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.066+0000 7fdc6ca1a700 1 --2- 172.21.15.32:0/2882637643 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5623119be060 0x5623119c2650 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fdc600a5130 tx=0x7fdc60009410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4833fc7ef93d4a5f server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:47:41.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.066+0000 7fdc477fe700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-05-03T18:47:41.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.066+0000 7fdc477fe700 1 -- 172.21.15.32:0/2882637643 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc600c2070 con 0x5623119be060 2024-05-03T18:47:41.085 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.066+0000 7fdc78d10b80 1 -- 172.21.15.32:0/2882637643 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc600ad460 con 0x5623119be060 2024-05-03T18:47:41.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.066+0000 7fdc477fe700 1 -- 172.21.15.32:0/2882637643 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8356+0+0 (secure 0 0 0) 0x7fdc600a9520 con 0x5623119be060 2024-05-03T18:47:41.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.066+0000 7fdc477fe700 1 -- 172.21.15.32:0/2882637643 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fdc600b1a80 con 0x5623119be060 2024-05-03T18:47:41.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.066+0000 7fdc78d10b80 10 client.15126 fetch_fsmap learned FSMap version 25 2024-05-03T18:47:41.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.066+0000 7fdc78d10b80 1 -- 172.21.15.32:0/2882637643 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5623119d0280 con 0x5623119be060 2024-05-03T18:47:41.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.067+0000 7fdc477fe700 1 -- 172.21.15.32:0/2882637643 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc600ad460 con 0x5623119be060 2024-05-03T18:47:41.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.067+0000 7fdc477fe700 1 -- 172.21.15.32:0/2882637643 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fdc600a8310 con 0x5623119be060 2024-05-03T18:47:41.086 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.067+0000 7fdc78d10b80 10 client.15126 fetch_fsmap finished waiting for FSMap version 25 2024-05-03T18:47:41.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.067+0000 7fdc78d10b80 10 client.15126 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-03T18:47:41.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.067+0000 7fdc78d10b80 20 client.15126 populate_metadata read hostname 'smithi032' 2024-05-03T18:47:41.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.067+0000 7fdc78d10b80 1 --2- 172.21.15.32:0/2882637643 >> [v2:172.21.15.113:6838/942050693,v1:172.21.15.113:6839/942050693] conn(0x5623119d15d0 0x5623119d37b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:41.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.067+0000 7fdc78d10b80 4 client.15126 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-03T18:47:41.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.067+0000 7fdc78d10b80 1 -- 172.21.15.32:0/2882637643 --> [v2:172.21.15.113:6838/942050693,v1:172.21.15.113:6839/942050693] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5623119192d0 con 0x5623119d15d0 2024-05-03T18:47:41.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.067+0000 7fdc67fff700 1 --2- 172.21.15.32:0/2882637643 >> [v2:172.21.15.113:6838/942050693,v1:172.21.15.113:6839/942050693] conn(0x5623119d15d0 0x5623119d37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:41.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.068+0000 7fdc67fff700 1 --2- 172.21.15.32:0/2882637643 >> [v2:172.21.15.113:6838/942050693,v1:172.21.15.113:6839/942050693] conn(0x5623119d15d0 0x5623119d37b0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd69ffc77e8adc3e server_cookie=b1ea44104e8ee1f5 in_seq=0 out_seq=0 2024-05-03T18:47:41.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.068+0000 7fdc477fe700 10 client.15126 ms_handle_connect on v2:172.21.15.113:6838/942050693 2024-05-03T18:47:41.087 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.068+0000 7fdc477fe700 1 -- 172.21.15.32:0/2882637643 <== mds.0 v2:172.21.15.113:6838/942050693 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5623119192d0 con 0x5623119d15d0 2024-05-03T18:47:41.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.068+0000 7fdc477fe700 10 client.15126 handle_command_reply: tid=0 2024-05-03T18:47:41.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.068+0000 7fdc78d10b80 1 client.15126 shutdown 2024-05-03T18:47:41.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.069+0000 7fdc78d10b80 1 -- 172.21.15.32:0/2882637643 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5623119be060 msgr2=0x5623119c2650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:41.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.069+0000 7fdc78d10b80 1 --2- 172.21.15.32:0/2882637643 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5623119be060 0x5623119c2650 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fdc600a5130 tx=0x7fdc60009410 comp rx=0 tx=0).stop 2024-05-03T18:47:41.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.069+0000 7fdc78d10b80 1 -- 172.21.15.32:0/2882637643 shutdown_connections 2024-05-03T18:47:41.088 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.069+0000 7fdc78d10b80 1 --2- 172.21.15.32:0/2882637643 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5623119be060 0x5623119c2650 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:41.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.069+0000 7fdc78d10b80 1 --2- 172.21.15.32:0/2882637643 >> [v2:172.21.15.113:6838/942050693,v1:172.21.15.113:6839/942050693] conn(0x5623119d15d0 0x5623119d37b0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:41.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.069+0000 7fdc477fe700 0 client.15126 ms_handle_reset on v2:172.21.15.113:6838/942050693 2024-05-03T18:47:41.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.069+0000 7fdc78d10b80 1 --2- 172.21.15.32:0/2882637643 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5623119b9480 0x5623119c4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:41.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.069+0000 7fdc78d10b80 1 --2- 172.21.15.32:0/2882637643 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5623119bd740 0x5623119c2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:41.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.069+0000 7fdc78d10b80 1 -- 172.21.15.32:0/2882637643 >> 172.21.15.32:0/2882637643 conn(0x5623118e5b20 msgr2=0x5623119bc3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:47:41.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.069+0000 7fdc78d10b80 1 -- 172.21.15.32:0/2882637643 shutdown_connections 2024-05-03T18:47:41.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.069+0000 7fdc78d10b80 1 -- 172.21.15.32:0/2882637643 wait complete. 2024-05-03T18:47:41.089 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.069+0000 7fdc78d10b80 20 client.15126 trim_cache size 0 max 16384 2024-05-03T18:47:41.092 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.090+0000 7fdc78d10b80 1 Processor -- start 2024-05-03T18:47:41.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.090+0000 7fdc78d10b80 1 -- start start 2024-05-03T18:47:41.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.090+0000 7fdc78d10b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5623119b9480 0x5623119bc380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:41.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.090+0000 7fdc78d10b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5623119bd740 0x5623119bc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:41.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.090+0000 7fdc78d10b80 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5623119be060 0x5623119bce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:41.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.090+0000 7fdc78d10b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5623119d1af0 con 0x5623119bd740 2024-05-03T18:47:41.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.090+0000 7fdc78d10b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5623119d21c0 con 0x5623119b9480 2024-05-03T18:47:41.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.090+0000 7fdc78d10b80 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5623119d2890 con 0x5623119be060 2024-05-03T18:47:41.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.090+0000 7fdc677fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5623119bd740 0x5623119bc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:41.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.090+0000 7fdc677fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5623119bd740 0x5623119bc8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34494/0 (socket says 172.21.15.32:34494) 2024-05-03T18:47:41.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.090+0000 7fdc677fe700 1 -- 172.21.15.32:0/4129901385 learned_addr learned my addr 172.21.15.32:0/4129901385 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:47:41.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.090+0000 7fdc67fff700 1 --2- 172.21.15.32:0/4129901385 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5623119b9480 0x5623119bc380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:41.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.090+0000 7fdc677fe700 1 -- 172.21.15.32:0/4129901385 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5623119be060 msgr2=0x5623119bce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:47:41.093 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.090+0000 7fdc677fe700 1 --2- 172.21.15.32:0/4129901385 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5623119be060 0x5623119bce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:41.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.090+0000 7fdc677fe700 1 -- 172.21.15.32:0/4129901385 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5623119b9480 msgr2=0x5623119bc380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:47:41.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.090+0000 7fdc677fe700 1 --2- 172.21.15.32:0/4129901385 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5623119b9480 0x5623119bc380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:41.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.090+0000 7fdc677fe700 1 -- 172.21.15.32:0/4129901385 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623119ceff0 con 0x5623119bd740 2024-05-03T18:47:41.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.091+0000 7fdc677fe700 1 --2- 172.21.15.32:0/4129901385 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5623119bd740 0x5623119bc8c0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fdc5c004590 tx=0x7fdc5c00baa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b76d9a4e9ba78f88 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:47:41.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.091+0000 7fdc4dffb700 1 -- 172.21.15.32:0/4129901385 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc5c003740 con 0x5623119bd740 2024-05-03T18:47:41.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.091+0000 7fdc4dffb700 1 -- 172.21.15.32:0/4129901385 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc5c003d90 con 0x5623119bd740 2024-05-03T18:47:41.094 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.091+0000 7fdc4dffb700 1 -- 172.21.15.32:0/4129901385 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc5c011e40 con 0x5623119bd740 2024-05-03T18:47:41.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.091+0000 7fdc78d10b80 1 -- 172.21.15.32:0/4129901385 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5623119bd740 msgr2=0x5623119bc8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:41.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.091+0000 7fdc78d10b80 1 --2- 172.21.15.32:0/4129901385 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5623119bd740 0x5623119bc8c0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fdc5c004590 tx=0x7fdc5c00baa0 comp rx=0 tx=0).stop 2024-05-03T18:47:41.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.091+0000 7fdc78d10b80 1 -- 172.21.15.32:0/4129901385 shutdown_connections 2024-05-03T18:47:41.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.091+0000 7fdc78d10b80 1 --2- 172.21.15.32:0/4129901385 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5623119be060 0x5623119bce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:41.098 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.091+0000 7fdc78d10b80 1 --2- 172.21.15.32:0/4129901385 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5623119b9480 0x5623119bc380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:41.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.091+0000 7fdc78d10b80 1 --2- 172.21.15.32:0/4129901385 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5623119bd740 0x5623119bc8c0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:41.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.091+0000 7fdc78d10b80 1 -- 172.21.15.32:0/4129901385 >> 172.21.15.32:0/4129901385 conn(0x5623118e5b20 msgr2=0x5623119c2c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:47:41.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.091+0000 7fdc78d10b80 1 -- 172.21.15.32:0/4129901385 shutdown_connections 2024-05-03T18:47:41.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.092+0000 7fdc78d10b80 1 -- 172.21.15.32:0/4129901385 wait complete. 2024-05-03T18:47:41.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.092+0000 7fdc78d10b80 1 -- start start 2024-05-03T18:47:41.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.092+0000 7fdc78d10b80 1 Processor -- start 2024-05-03T18:47:41.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.092+0000 7fdc78d10b80 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5623119b9480 0x5623119dcc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:41.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.092+0000 7fdc78d10b80 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5623119bd740 0x5623119dd170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:41.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.092+0000 7fdc78d10b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5623119be060 0x5623119d5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:41.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.092+0000 7fdc78d10b80 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x5623119d3630 con 0x5623119bd740 2024-05-03T18:47:41.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.092+0000 7fdc78d10b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5623119d2f60 con 0x5623119be060 2024-05-03T18:47:41.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.092+0000 7fdc78d10b80 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5623119d3d00 con 0x5623119b9480 2024-05-03T18:47:41.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.092+0000 7fdc78d10b80 10 client.0 _ll_register_callbacks cb 0x56231189f560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-03T18:47:41.099 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.092+0000 7fdc677fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5623119bd740 0x5623119dd170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:41.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.092+0000 7fdc677fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5623119bd740 0x5623119dd170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34502/0 (socket says 172.21.15.32:34502) 2024-05-03T18:47:41.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.092+0000 7fdc677fe700 1 -- 172.21.15.32:0/1459500588 learned_addr learned my addr 172.21.15.32:0/1459500588 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:47:41.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.093+0000 7fdc67fff700 1 --2- 172.21.15.32:0/1459500588 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5623119b9480 0x5623119dcc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:41.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.093+0000 7fdc6ca1a700 1 --2- 172.21.15.32:0/1459500588 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5623119be060 0x5623119d5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:41.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.093+0000 7fdc677fe700 1 -- 172.21.15.32:0/1459500588 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5623119b9480 msgr2=0x5623119dcc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:47:41.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.093+0000 7fdc677fe700 1 --2- 172.21.15.32:0/1459500588 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5623119b9480 0x5623119dcc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:41.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.093+0000 7fdc677fe700 1 -- 172.21.15.32:0/1459500588 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5623119be060 msgr2=0x5623119d5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-03T18:47:41.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.093+0000 7fdc677fe700 1 --2- 172.21.15.32:0/1459500588 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5623119be060 0x5623119d5bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:41.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.093+0000 7fdc677fe700 1 -- 172.21.15.32:0/1459500588 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623119d61e0 con 0x5623119bd740 2024-05-03T18:47:41.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.093+0000 7fdc67fff700 1 --2- 172.21.15.32:0/1459500588 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5623119b9480 0x5623119dcc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T18:47:41.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.093+0000 7fdc6ca1a700 1 --2- 172.21.15.32:0/1459500588 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5623119be060 0x5623119d5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T18:47:41.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.093+0000 7fdc677fe700 1 --2- 172.21.15.32:0/1459500588 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5623119bd740 0x5623119dd170 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fdc5c0046c0 tx=0x7fdc5c011de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f239ddb5a14ae32 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:47:41.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.093+0000 7fdc477fe700 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2024-05-03T18:47:41.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.093+0000 7fdc477fe700 1 -- 172.21.15.32:0/1459500588 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc5c003950 con 0x5623119bd740 2024-05-03T18:47:41.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.093+0000 7fdc477fe700 1 -- 172.21.15.32:0/1459500588 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8356+0+0 (secure 0 0 0) 0x7fdc5c017040 con 0x5623119bd740 2024-05-03T18:47:41.100 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.093+0000 7fdc78d10b80 1 -- 172.21.15.32:0/1459500588 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc5c003d90 con 0x5623119bd740 2024-05-03T18:47:41.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.093+0000 7fdc477fe700 1 -- 172.21.15.32:0/1459500588 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fdc5c003af0 con 0x5623119bd740 2024-05-03T18:47:41.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.093+0000 7fdc78d10b80 10 client.15132 fetch_fsmap learned FSMap version 25 2024-05-03T18:47:41.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.093+0000 7fdc78d10b80 1 -- 172.21.15.32:0/1459500588 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x562311a3e0b0 con 0x5623119bd740 2024-05-03T18:47:41.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.093+0000 7fdc477fe700 1 -- 172.21.15.32:0/1459500588 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc5c003d90 con 0x5623119bd740 2024-05-03T18:47:41.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.094+0000 7fdc477fe700 1 -- 172.21.15.32:0/1459500588 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fdc5c01b4b0 con 0x5623119bd740 2024-05-03T18:47:41.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.094+0000 7fdc78d10b80 10 client.15132 fetch_fsmap finished waiting for FSMap version 25 2024-05-03T18:47:41.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.094+0000 7fdc78d10b80 10 client.15132 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-03T18:47:41.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.094+0000 7fdc78d10b80 20 client.15132 populate_metadata read hostname 'smithi032' 2024-05-03T18:47:41.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.094+0000 7fdc78d10b80 1 --2- 172.21.15.32:0/1459500588 >> [v2:172.21.15.113:6838/942050693,v1:172.21.15.113:6839/942050693] conn(0x5623119d15d0 0x562311a41780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:41.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.094+0000 7fdc78d10b80 4 client.15132 mds_command: new command op to 24478 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-03T18:47:41.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.094+0000 7fdc78d10b80 1 -- 172.21.15.32:0/1459500588 --> [v2:172.21.15.113:6838/942050693,v1:172.21.15.113:6839/942050693] -- command(tid 0: {"prefix": "status"}) -- 0x562311913370 con 0x5623119d15d0 2024-05-03T18:47:41.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.094+0000 7fdc67fff700 1 --2- 172.21.15.32:0/1459500588 >> [v2:172.21.15.113:6838/942050693,v1:172.21.15.113:6839/942050693] conn(0x5623119d15d0 0x562311a41780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:41.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.094+0000 7fdc67fff700 1 --2- 172.21.15.32:0/1459500588 >> [v2:172.21.15.113:6838/942050693,v1:172.21.15.113:6839/942050693] conn(0x5623119d15d0 0x562311a41780 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81bdb45b94234593 server_cookie=5667fb9bafd0ee53 in_seq=0 out_seq=0 2024-05-03T18:47:41.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.094+0000 7fdc477fe700 10 client.15132 ms_handle_connect on v2:172.21.15.113:6838/942050693 2024-05-03T18:47:41.101 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.095+0000 7fdc477fe700 1 -- 172.21.15.32:0/1459500588 <== mds.0 v2:172.21.15.113:6838/942050693 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x562311913370 con 0x5623119d15d0 2024-05-03T18:47:41.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.095+0000 7fdc477fe700 10 client.15132 handle_command_reply: tid=0 2024-05-03T18:47:41.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.095+0000 7fdc78d10b80 1 client.15132 shutdown 2024-05-03T18:47:41.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.095+0000 7fdc78d10b80 1 -- 172.21.15.32:0/1459500588 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5623119bd740 msgr2=0x5623119dd170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:41.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.095+0000 7fdc78d10b80 1 --2- 172.21.15.32:0/1459500588 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5623119bd740 0x5623119dd170 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fdc5c0046c0 tx=0x7fdc5c011de0 comp rx=0 tx=0).stop 2024-05-03T18:47:41.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.095+0000 7fdc78d10b80 1 -- 172.21.15.32:0/1459500588 shutdown_connections 2024-05-03T18:47:41.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.095+0000 7fdc78d10b80 1 --2- 172.21.15.32:0/1459500588 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5623119be060 0x5623119d5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:41.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.095+0000 7fdc78d10b80 1 --2- 172.21.15.32:0/1459500588 >> [v2:172.21.15.113:6838/942050693,v1:172.21.15.113:6839/942050693] conn(0x5623119d15d0 0x562311a41780 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:41.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.095+0000 7fdc78d10b80 1 --2- 172.21.15.32:0/1459500588 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5623119b9480 0x5623119dcc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:41.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.095+0000 7fdc78d10b80 1 --2- 172.21.15.32:0/1459500588 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5623119bd740 0x5623119dd170 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:41.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.095+0000 7fdc477fe700 0 client.15132 ms_handle_reset on v2:172.21.15.113:6838/942050693 2024-05-03T18:47:41.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.095+0000 7fdc78d10b80 1 -- 172.21.15.32:0/1459500588 >> 172.21.15.32:0/1459500588 conn(0x5623118e5b20 msgr2=0x5623119cdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:47:41.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.095+0000 7fdc78d10b80 1 -- 172.21.15.32:0/1459500588 shutdown_connections 2024-05-03T18:47:41.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.095+0000 7fdc78d10b80 1 -- 172.21.15.32:0/1459500588 wait complete. 2024-05-03T18:47:41.102 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.095+0000 7fdc78d10b80 20 client.15132 trim_cache size 0 max 16384 2024-05-03T18:47:41.103 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2024-05-03T18:47:41.103 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_fsid": "f500455e-097b-11ef-bc97-c7b262605968", 2024-05-03T18:47:41.103 INFO:teuthology.orchestra.run.smithi032.stdout: "whoami": 0, 2024-05-03T18:47:41.103 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 24478, 2024-05-03T18:47:41.103 INFO:teuthology.orchestra.run.smithi032.stdout: "want_state": "up:active", 2024-05-03T18:47:41.103 INFO:teuthology.orchestra.run.smithi032.stdout: "state": "up:active", 2024-05-03T18:47:41.103 INFO:teuthology.orchestra.run.smithi032.stdout: "fs_name": "cephfs", 2024-05-03T18:47:41.103 INFO:teuthology.orchestra.run.smithi032.stdout: "rank_uptime": 169.675757721, 2024-05-03T18:47:41.103 INFO:teuthology.orchestra.run.smithi032.stdout: "mdsmap_epoch": 25, 2024-05-03T18:47:41.103 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch": 82, 2024-05-03T18:47:41.103 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch_barrier": 78, 2024-05-03T18:47:41.104 INFO:teuthology.orchestra.run.smithi032.stdout: "uptime": 193.365210023 2024-05-03T18:47:41.104 INFO:teuthology.orchestra.run.smithi032.stdout:} 2024-05-03T18:47:41.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.096+0000 7fdc6e47d700 1 -- 172.21.15.32:0/4212755952 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fdc500b2910 msgr2=0x7fdc500b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:41.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.096+0000 7fdc6e47d700 1 --2- 172.21.15.32:0/4212755952 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fdc500b2910 0x7fdc500b4dd0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fdc58003800 tx=0x7fdc58005fb0 comp rx=0 tx=0).stop 2024-05-03T18:47:41.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.096+0000 7fdc6e47d700 1 -- 172.21.15.32:0/4212755952 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc68108870 msgr2=0x7fdc68199960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:41.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.096+0000 7fdc6e47d700 1 --2- 172.21.15.32:0/4212755952 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc68108870 0x7fdc68199960 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fdc60007ee0 tx=0x7fdc6000d3b0 comp rx=0 tx=0).stop 2024-05-03T18:47:41.104 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.096+0000 7fdc6e47d700 1 -- 172.21.15.32:0/4212755952 shutdown_connections 2024-05-03T18:47:41.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.096+0000 7fdc6e47d700 1 --2- 172.21.15.32:0/4212755952 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fdc500b2910 0x7fdc500b4dd0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:41.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.096+0000 7fdc6e47d700 1 --2- 172.21.15.32:0/4212755952 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc68108870 0x7fdc68199960 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:41.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.096+0000 7fdc6e47d700 1 --2- 172.21.15.32:0/4212755952 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdc6806bdd0 0x7fdc6819f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:41.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.096+0000 7fdc6e47d700 1 --2- 172.21.15.32:0/4212755952 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc6806c780 0x7fdc6819fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:41.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.096+0000 7fdc6e47d700 1 -- 172.21.15.32:0/4212755952 >> 172.21.15.32:0/4212755952 conn(0x7fdc68079350 msgr2=0x7fdc68100a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:47:41.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.096+0000 7fdc6e47d700 1 -- 172.21.15.32:0/4212755952 shutdown_connections 2024-05-03T18:47:41.105 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:41.096+0000 7fdc6e47d700 1 -- 172.21.15.32:0/4212755952 wait complete. 2024-05-03T18:47:41.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:47:41 smithi045 ceph-mon[33299]: pgmap v369: 97 pgs: 97 active+clean; 631 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:47:41.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:47:41 smithi113 ceph-mon[33540]: pgmap v369: 97 pgs: 97 active+clean; 631 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:47:41.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:47:41 smithi032 ceph-mon[29763]: pgmap v369: 97 pgs: 97 active+clean; 631 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:47:41.869 INFO:teuthology.run_tasks:Running task sequential... 2024-05-03T18:47:41.879 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-05-03T18:47:41.879 INFO:teuthology.task.sequential:In sequential, running task print... 2024-05-03T18:47:41.880 INFO:teuthology.task.print:Enabling mgr modules 2024-05-03T18:47:41.880 INFO:teuthology.run_tasks:Running task kclient... 2024-05-03T18:47:41.903 INFO:tasks.kclient:Mounting kernel clients... 2024-05-03T18:47:41.904 INFO:tasks.kclient:config is {'client.0': {}} 2024-05-03T18:47:41.904 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-03T18:47:41.904 DEBUG:teuthology.orchestra.run.smithi032:> ip netns list 2024-05-03T18:47:41.923 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-03T18:47:41.923 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link delete ceph-brx 2024-05-03T18:47:41.996 INFO:teuthology.orchestra.run.smithi032.stderr:Cannot find device "ceph-brx" 2024-05-03T18:47:41.997 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-03T18:47:41.997 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=crc', 'wsync'], 'syntax': 'v1', 'mount_subvol_num': 1} 2024-05-03T18:47:41.998 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-03T18:47:41.998 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-03T18:47:41.998 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid f500455e-097b-11ef-bc97-c7b262605968 -- ceph fs ls 2024-05-03T18:47:42.289 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T18:47:43.576 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:47:43 smithi032 ceph-mon[29763]: pgmap v370: 97 pgs: 97 active+clean; 631 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:47:43.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:47:43 smithi045 ceph-mon[33299]: pgmap v370: 97 pgs: 97 active+clean; 631 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:47:43.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:47:43 smithi113 ceph-mon[33540]: pgmap v370: 97 pgs: 97 active+clean; 631 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:47:43.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:43.932+0000 7f324e12b700 1 -- 172.21.15.32:0/3929103336 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3248102d60 msgr2=0x7f324810f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:43.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:43.932+0000 7f324e12b700 1 --2- 172.21.15.32:0/3929103336 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3248102d60 0x7f324810f250 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f324000b0a0 tx=0x7f324000b3b0 comp rx=0 tx=0).stop 2024-05-03T18:47:43.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:43.932+0000 7f324e12b700 1 -- 172.21.15.32:0/3929103336 shutdown_connections 2024-05-03T18:47:43.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:43.932+0000 7f324e12b700 1 --2- 172.21.15.32:0/3929103336 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3248102d60 0x7f324810f250 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:43.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:43.932+0000 7f324e12b700 1 --2- 172.21.15.32:0/3929103336 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3248108410 0x7f32481087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:43.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:43.932+0000 7f324e12b700 1 --2- 172.21.15.32:0/3929103336 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f32481023a0 0x7f3248102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:43.935 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:43.932+0000 7f324e12b700 1 -- 172.21.15.32:0/3929103336 >> 172.21.15.32:0/3929103336 conn(0x7f3248079350 msgr2=0x7f3248079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:47:43.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:43.932+0000 7f324e12b700 1 -- 172.21.15.32:0/3929103336 shutdown_connections 2024-05-03T18:47:43.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:43.932+0000 7f324e12b700 1 -- 172.21.15.32:0/3929103336 wait complete. 2024-05-03T18:47:43.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:43.933+0000 7f324e12b700 1 Processor -- start 2024-05-03T18:47:43.936 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:43.933+0000 7f324e12b700 1 -- start start 2024-05-03T18:47:43.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:43.933+0000 7f324e12b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32481023a0 0x7f324819e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:43.937 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:43.933+0000 7f324e12b700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3248102d60 0x7f324819ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:43.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:43.933+0000 7f324e12b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3248108410 0x7f32481988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:43.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:43.933+0000 7f324e12b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f32481125b0 con 0x7f32481023a0 2024-05-03T18:47:43.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:43.933+0000 7f324e12b700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3248112430 con 0x7f3248108410 2024-05-03T18:47:43.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:43.933+0000 7f324e12b700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f3248112730 con 0x7f3248102d60 2024-05-03T18:47:43.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:43.933+0000 7f32477fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32481023a0 0x7f324819e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:43.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:43.934+0000 7f32477fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32481023a0 0x7f324819e730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46566/0 (socket says 172.21.15.32:46566) 2024-05-03T18:47:43.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:43.934+0000 7f32477fe700 1 -- 172.21.15.32:0/492539080 learned_addr learned my addr 172.21.15.32:0/492539080 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:47:43.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:43.934+0000 7f3246ffd700 1 --2- 172.21.15.32:0/492539080 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3248102d60 0x7f324819ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:43.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:43.934+0000 7f3247fff700 1 --2- 172.21.15.32:0/492539080 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3248108410 0x7f32481988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:43.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:43.934+0000 7f32477fe700 1 -- 172.21.15.32:0/492539080 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3248102d60 msgr2=0x7f324819ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:43.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:43.934+0000 7f32477fe700 1 --2- 172.21.15.32:0/492539080 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3248102d60 0x7f324819ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:43.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:43.934+0000 7f32477fe700 1 -- 172.21.15.32:0/492539080 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3248108410 msgr2=0x7f32481988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:43.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:43.934+0000 7f32477fe700 1 --2- 172.21.15.32:0/492539080 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3248108410 0x7f32481988f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:43.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:43.934+0000 7f32477fe700 1 -- 172.21.15.32:0/492539080 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3248199050 con 0x7f32481023a0 2024-05-03T18:47:43.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:43.934+0000 7f32477fe700 1 --2- 172.21.15.32:0/492539080 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32481023a0 0x7f324819e730 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f323c00b530 tx=0x7f323c00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:47:43.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:43.934+0000 7f3244ff9700 1 -- 172.21.15.32:0/492539080 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f323c015020 con 0x7f32481023a0 2024-05-03T18:47:43.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:43.934+0000 7f324e12b700 1 -- 172.21.15.32:0/492539080 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3248199340 con 0x7f32481023a0 2024-05-03T18:47:43.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:43.934+0000 7f3244ff9700 1 -- 172.21.15.32:0/492539080 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f323c011ce0 con 0x7f32481023a0 2024-05-03T18:47:43.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:43.934+0000 7f3244ff9700 1 -- 172.21.15.32:0/492539080 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f323c010dd0 con 0x7f32481023a0 2024-05-03T18:47:43.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:43.934+0000 7f324e12b700 1 -- 172.21.15.32:0/492539080 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32481a5600 con 0x7f32481023a0 2024-05-03T18:47:43.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:43.935+0000 7f324e12b700 1 -- 172.21.15.32:0/492539080 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f324806a190 con 0x7f32481023a0 2024-05-03T18:47:43.939 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:43.938+0000 7f3244ff9700 1 -- 172.21.15.32:0/492539080 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f323c018020 con 0x7f32481023a0 2024-05-03T18:47:43.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:43.938+0000 7f3244ff9700 1 --2- 172.21.15.32:0/492539080 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f32300b2910 0x7f32300b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:43.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:43.938+0000 7f3246ffd700 1 --2- 172.21.15.32:0/492539080 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f32300b2910 0x7f32300b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:43.940 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:43.938+0000 7f3244ff9700 1 -- 172.21.15.32:0/492539080 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8356+0+0 (secure 0 0 0) 0x7f323c0e1f90 con 0x7f32481023a0 2024-05-03T18:47:43.941 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:43.939+0000 7f3246ffd700 1 --2- 172.21.15.32:0/492539080 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f32300b2910 0x7f32300b4dd0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f3248079990 tx=0x7f323800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:47:43.944 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:43.942+0000 7f3244ff9700 1 -- 172.21.15.32:0/492539080 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f323c01f020 con 0x7f32481023a0 2024-05-03T18:47:44.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:44.205+0000 7f324e12b700 1 -- 172.21.15.32:0/492539080 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f3248199e20 con 0x7f32481023a0 2024-05-03T18:47:44.207 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:44.205+0000 7f3244ff9700 1 -- 172.21.15.32:0/492539080 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v25) ==== 53+0+73 (secure 0 0 0) 0x7f323c0a9be0 con 0x7f32481023a0 2024-05-03T18:47:44.207 INFO:teuthology.orchestra.run.smithi032.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-03T18:47:44.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:44.208+0000 7f324e12b700 1 -- 172.21.15.32:0/492539080 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f32300b2910 msgr2=0x7f32300b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:44.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:44.208+0000 7f324e12b700 1 --2- 172.21.15.32:0/492539080 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f32300b2910 0x7f32300b4dd0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f3248079990 tx=0x7f323800b410 comp rx=0 tx=0).stop 2024-05-03T18:47:44.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:44.208+0000 7f324e12b700 1 -- 172.21.15.32:0/492539080 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32481023a0 msgr2=0x7f324819e730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:44.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:44.208+0000 7f324e12b700 1 --2- 172.21.15.32:0/492539080 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32481023a0 0x7f324819e730 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f323c00b530 tx=0x7f323c00b840 comp rx=0 tx=0).stop 2024-05-03T18:47:44.210 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:44.208+0000 7f324e12b700 1 -- 172.21.15.32:0/492539080 shutdown_connections 2024-05-03T18:47:44.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:44.208+0000 7f324e12b700 1 --2- 172.21.15.32:0/492539080 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f32300b2910 0x7f32300b4dd0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:44.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:44.208+0000 7f324e12b700 1 --2- 172.21.15.32:0/492539080 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3248108410 0x7f32481988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:44.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:44.208+0000 7f324e12b700 1 --2- 172.21.15.32:0/492539080 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32481023a0 0x7f324819e730 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:44.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:44.208+0000 7f324e12b700 1 --2- 172.21.15.32:0/492539080 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3248102d60 0x7f324819ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:44.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:44.208+0000 7f324e12b700 1 -- 172.21.15.32:0/492539080 >> 172.21.15.32:0/492539080 conn(0x7f3248079350 msgr2=0x7f3248100f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:47:44.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:44.208+0000 7f324e12b700 1 -- 172.21.15.32:0/492539080 shutdown_connections 2024-05-03T18:47:44.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:44.208+0000 7f324e12b700 1 -- 172.21.15.32:0/492539080 wait complete. 2024-05-03T18:47:44.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:47:44 smithi032 ceph-mon[29763]: from='client.? 172.21.15.32:0/492539080' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-03T18:47:44.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:47:44 smithi045 ceph-mon[33299]: from='client.? 172.21.15.32:0/492539080' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-03T18:47:44.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:47:44 smithi113 ceph-mon[33540]: from='client.? 172.21.15.32:0/492539080' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-03T18:47:44.984 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-03T18:47:44.985 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-03T18:47:44.985 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi032.front.sepia.ceph.com 2024-05-03T18:47:44.985 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-03T18:47:44.985 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-03T18:47:44.985 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-03T18:47:44.985 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-03T18:47:44.985 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-03T18:47:44.985 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-03T18:47:44.985 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-03T18:47:44.985 DEBUG:teuthology.orchestra.run.smithi032:> ip addr 2024-05-03T18:47:45.003 INFO:teuthology.orchestra.run.smithi032.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-03T18:47:45.004 INFO:teuthology.orchestra.run.smithi032.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-03T18:47:45.004 INFO:teuthology.orchestra.run.smithi032.stdout: inet 127.0.0.1/8 scope host lo 2024-05-03T18:47:45.004 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-05-03T18:47:45.004 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 ::1/128 scope host 2024-05-03T18:47:45.004 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-05-03T18:47:45.004 INFO:teuthology.orchestra.run.smithi032.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-03T18:47:45.004 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0c:c4:7a:6b:fd:0c brd ff:ff:ff:ff:ff:ff 2024-05-03T18:47:45.004 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp7s0f0 2024-05-03T18:47:45.004 INFO:teuthology.orchestra.run.smithi032.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-03T18:47:45.004 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0c:c4:7a:6b:fd:0d brd ff:ff:ff:ff:ff:ff 2024-05-03T18:47:45.004 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp7s0f1 2024-05-03T18:47:45.004 INFO:teuthology.orchestra.run.smithi032.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-03T18:47:45.004 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether ac:1f:6b:f8:26:02 brd ff:ff:ff:ff:ff:ff 2024-05-03T18:47:45.005 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp1s0f0 2024-05-03T18:47:45.005 INFO:teuthology.orchestra.run.smithi032.stdout: inet 172.21.15.32/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-05-03T18:47:45.005 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft 41998sec preferred_lft 41998sec 2024-05-03T18:47:45.005 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::ae1f:6bff:fef8:2602/64 scope link noprefixroute 2024-05-03T18:47:45.005 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2024-05-03T18:47:45.005 INFO:teuthology.orchestra.run.smithi032.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-03T18:47:45.005 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether ac:1f:6b:f8:26:03 brd ff:ff:ff:ff:ff:ff 2024-05-03T18:47:45.005 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp1s0f1 2024-05-03T18:47:45.005 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-03T18:47:45.006 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-03T18:47:45.006 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2024-05-03T18:47:45.006 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link add name ceph-brx type bridge 2024-05-03T18:47:45.006 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip addr flush dev ceph-brx 2024-05-03T18:47:45.006 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link set ceph-brx up 2024-05-03T18:47:45.006 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-03T18:47:45.006 DEBUG:teuthology.orchestra.run.smithi032:> ') 2024-05-03T18:47:45.097 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:45 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-03T18:47:45.185 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-03T18:47:45.191 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-03T18:47:45.191 DEBUG:teuthology.orchestra.run.smithi032:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-03T18:47:45.216 INFO:teuthology.orchestra.run.smithi032.stdout:1 2024-05-03T18:47:45.217 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-03T18:47:45.218 DEBUG:teuthology.orchestra.run.smithi032:> route 2024-05-03T18:47:45.289 INFO:teuthology.orchestra.run.smithi032.stdout:Kernel IP routing table 2024-05-03T18:47:45.289 INFO:teuthology.orchestra.run.smithi032.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-03T18:47:45.289 INFO:teuthology.orchestra.run.smithi032.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-03T18:47:45.289 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-03T18:47:45.289 INFO:teuthology.orchestra.run.smithi032.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-03T18:47:45.289 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-03T18:47:45.290 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2024-05-03T18:47:45.290 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-03T18:47:45.290 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-03T18:47:45.290 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-03T18:47:45.290 DEBUG:teuthology.orchestra.run.smithi032:> ') 2024-05-03T18:47:45.378 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:45 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-03T18:47:45.473 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:47:45 smithi032 ceph-mon[29763]: pgmap v371: 97 pgs: 97 active+clean; 631 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:47:45.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-03T18:47:45.551 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-03T18:47:45.551 DEBUG:teuthology.orchestra.run.smithi032:> ip netns list 2024-05-03T18:47:45.567 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-03T18:47:45.567 DEBUG:teuthology.orchestra.run.smithi032:> ip netns list-id 2024-05-03T18:47:45.622 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-03T18:47:45.623 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2024-05-03T18:47:45.623 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-03T18:47:45.623 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-05-03T18:47:45.623 DEBUG:teuthology.orchestra.run.smithi032:> ') 2024-05-03T18:47:45.711 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:45 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-03T18:47:45.734 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-03T18:47:45.740 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-05-03T18:47:45.741 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-03T18:47:45.741 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2024-05-03T18:47:45.741 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-05-03T18:47:45.741 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-03T18:47:45.741 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-05-03T18:47:45.741 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-05-03T18:47:45.741 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-05-03T18:47:45.741 DEBUG:teuthology.orchestra.run.smithi032:> ') 2024-05-03T18:47:45.747 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:47:45 smithi045 ceph-mon[33299]: pgmap v371: 97 pgs: 97 active+clean; 631 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:47:45.790 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:45 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-03T18:47:45.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:47:45 smithi113 ceph-mon[33540]: pgmap v371: 97 pgs: 97 active+clean; 631 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:47:45.861 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-03T18:47:45.868 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-03T18:47:45.868 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2024-05-03T18:47:45.868 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link set brx.0 up 2024-05-03T18:47:45.868 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link set dev brx.0 master ceph-brx 2024-05-03T18:47:45.868 DEBUG:teuthology.orchestra.run.smithi032:> ') 2024-05-03T18:47:45.917 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:45 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-03T18:47:45.938 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-03T18:47:45.947 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-03T18:47:45.947 DEBUG:teuthology.orchestra.run.smithi032:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-03T18:47:45.962 INFO:teuthology.orchestra.run.smithi032.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-03T18:47:45.962 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-03T18:47:45.962 DEBUG:teuthology.orchestra.run.smithi032:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-03T18:47:46.017 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-05-03T18:47:46.017 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-05-03T18:47:46.017 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/qa/sv_1/eb69fd66-9cd9-414a-89e3-85fc2655cfa5 2024-05-03T18:47:46.017 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-03T18:47:46.018 DEBUG:teuthology.orchestra.run.smithi032:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/qa/sv_1/eb69fd66-9cd9-414a-89e3-85fc2655cfa5 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync 2024-05-03T18:47:46.112 INFO:teuthology.orchestra.run.smithi032.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync 2024-05-03T18:47:46.112 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync". 2024-05-03T18:47:46.112 INFO:teuthology.orchestra.run.smithi032.stdout:invalid new device string format 2024-05-03T18:47:46.217 INFO:teuthology.orchestra.run.smithi032.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync 2024-05-03T18:47:46.217 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync". 2024-05-03T18:47:46.217 INFO:teuthology.orchestra.run.smithi032.stdout:invalid new device string format 2024-05-03T18:47:46.217 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: resolved to: "172.21.15.32:3300,172.21.15.45:3300,172.21.15.113:3300" 2024-05-03T18:47:46.217 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.32:3300,172.21.15.45:3300,172.21.15.113:3300:/volumes/qa/sv_1/eb69fd66-9cd9-414a-89e3-85fc2655cfa5 2024-05-03T18:47:46.217 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync,key=0,fsid=f500455e-097b-11ef-bc97-c7b262605968" will pass to kernel 2024-05-03T18:47:46.218 INFO:teuthology.orchestra.run.smithi032.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-05-03T18:47:46.218 INFO:teuthology.orchestra.run.smithi032.stdout: You just mounted a file system that supports labels which does not 2024-05-03T18:47:46.218 INFO:teuthology.orchestra.run.smithi032.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-03T18:47:46.218 INFO:teuthology.orchestra.run.smithi032.stdout: applications will generate AVC messages and not be allowed access to 2024-05-03T18:47:46.218 INFO:teuthology.orchestra.run.smithi032.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-03T18:47:46.220 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-03T18:47:46.220 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-03T18:47:46.220 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-03T18:47:46.252 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-03T18:47:46.344 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:46 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-03T18:47:46.346 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-03T18:47:46.351 INFO:teuthology.orchestra.run.smithi032.stdout:client24776 2024-05-03T18:47:46.352 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-05-03T18:47:46.352 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-03T18:47:46.369 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-03T18:47:46.458 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:46 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-03T18:47:46.460 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-03T18:47:46.466 INFO:teuthology.orchestra.run.smithi032.stdout:client24776 2024-05-03T18:47:46.466 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-03T18:47:46.516 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:46 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-03T18:47:46.518 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-03T18:47:46.524 INFO:teuthology.orchestra.run.smithi032.stdout:client24776 2024-05-03T18:47:46.524 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/f500455e-097b-11ef-bc97-c7b262605968.client24776/status') 2024-05-03T18:47:46.573 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:46 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-03T18:47:46.583 INFO:teuthology.orchestra.run.smithi032.stdout:instance: client.24776 (3)172.21.15.32:0/304855475 2024-05-03T18:47:46.583 INFO:teuthology.orchestra.run.smithi032.stdout:blocklisted: false 2024-05-03T18:47:46.583 INFO:teuthology.orchestra.run.smithi032.stderr:0+1 records in 2024-05-03T18:47:46.583 INFO:teuthology.orchestra.run.smithi032.stderr:0+1 records out 2024-05-03T18:47:46.583 INFO:teuthology.orchestra.run.smithi032.stderr:70 bytes copied, 2.7439e-05 s, 2.6 MB/s 2024-05-03T18:47:46.584 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-03T18:47:46.591 INFO:teuthology.run_tasks:Running task exec... 2024-05-03T18:47:46.607 INFO:teuthology.task.exec:Executing custom commands... 2024-05-03T18:47:46.607 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi032.front.sepia.ceph.com 2024-05-03T18:47:46.607 DEBUG:teuthology.orchestra.run.smithi032:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-05-03T18:47:46.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:46.715+0000 7f18a82ea640 1 -- 172.21.15.32:0/721238444 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18a0101910 msgr2=0x7f18a0101d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:46.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:46.715+0000 7f18a82ea640 1 --2- 172.21.15.32:0/721238444 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18a0101910 0x7f18a0101d30 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f189c009ca0 tx=0x7f189c02a8c0 comp rx=0 tx=0).stop 2024-05-03T18:47:46.720 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:46.715+0000 7f18a82ea640 1 -- 172.21.15.32:0/721238444 shutdown_connections 2024-05-03T18:47:46.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:46.715+0000 7f18a82ea640 1 --2- 172.21.15.32:0/721238444 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18a01034c0 0x7f18a0109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:46.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:46.715+0000 7f18a82ea640 1 --2- 172.21.15.32:0/721238444 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18a0102b00 0x7f18a0102f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:46.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:46.715+0000 7f18a82ea640 1 --2- 172.21.15.32:0/721238444 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18a0101910 0x7f18a0101d30 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:46.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:46.715+0000 7f18a82ea640 1 -- 172.21.15.32:0/721238444 >> 172.21.15.32:0/721238444 conn(0x7f18a00fce80 msgr2=0x7f18a00ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:47:46.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:46.715+0000 7f18a82ea640 1 -- 172.21.15.32:0/721238444 shutdown_connections 2024-05-03T18:47:46.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:46.715+0000 7f18a82ea640 1 -- 172.21.15.32:0/721238444 wait complete. 2024-05-03T18:47:46.721 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:46.716+0000 7f18a82ea640 1 Processor -- start 2024-05-03T18:47:46.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:46.716+0000 7f18a82ea640 1 -- start start 2024-05-03T18:47:46.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:46.716+0000 7f18a82ea640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18a0101910 0x7f18a006ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:46.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:46.716+0000 7f18a82ea640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18a0102b00 0x7f18a019b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:46.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:46.716+0000 7f18a82ea640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18a01034c0 0x7f18a019bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:46.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:46.716+0000 7f18a82ea640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f18a010d460 con 0x7f18a01034c0 2024-05-03T18:47:46.722 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:46.716+0000 7f18a82ea640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f18a010d2e0 con 0x7f18a0102b00 2024-05-03T18:47:46.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:46.716+0000 7f18a82ea640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f18a010d5e0 con 0x7f18a0101910 2024-05-03T18:47:46.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:46.716+0000 7f18a6887640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18a01034c0 0x7f18a019bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:46.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:46.716+0000 7f18a6887640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18a01034c0 0x7f18a019bd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46586/0 (socket says 172.21.15.32:46586) 2024-05-03T18:47:46.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:46.716+0000 7f18a6887640 1 -- 172.21.15.32:0/1269696407 learned_addr learned my addr 172.21.15.32:0/1269696407 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:47:46.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:46.716+0000 7f18a6086640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18a0101910 0x7f18a006ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:46.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:46.716+0000 7f18a5885640 1 --2- 172.21.15.32:0/1269696407 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18a0102b00 0x7f18a019b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:46.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:46.716+0000 7f18a6887640 1 -- 172.21.15.32:0/1269696407 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18a0101910 msgr2=0x7f18a006ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:46.723 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:46.716+0000 7f18a6887640 1 --2- 172.21.15.32:0/1269696407 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18a0101910 0x7f18a006ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:46.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:46.716+0000 7f18a6887640 1 -- 172.21.15.32:0/1269696407 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18a0102b00 msgr2=0x7f18a019b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:46.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:46.716+0000 7f18a6887640 1 --2- 172.21.15.32:0/1269696407 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18a0102b00 0x7f18a019b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:46.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:46.716+0000 7f18a6887640 1 -- 172.21.15.32:0/1269696407 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18a019c5d0 con 0x7f18a01034c0 2024-05-03T18:47:46.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:46.717+0000 7f18a6887640 1 --2- 172.21.15.32:0/1269696407 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18a01034c0 0x7f18a019bd10 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f189800e970 tx=0x7f189800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:47:46.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:46.717+0000 7f18977fe640 1 -- 172.21.15.32:0/1269696407 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f189800ccc0 con 0x7f18a01034c0 2024-05-03T18:47:46.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:46.717+0000 7f18977fe640 1 -- 172.21.15.32:0/1269696407 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f189800ce60 con 0x7f18a01034c0 2024-05-03T18:47:46.724 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:46.717+0000 7f18977fe640 1 -- 172.21.15.32:0/1269696407 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1898010430 con 0x7f18a01034c0 2024-05-03T18:47:46.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:46.717+0000 7f18a82ea640 1 -- 172.21.15.32:0/1269696407 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18a019c8c0 con 0x7f18a01034c0 2024-05-03T18:47:46.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:46.717+0000 7f18a82ea640 1 -- 172.21.15.32:0/1269696407 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18a019cdc0 con 0x7f18a01034c0 2024-05-03T18:47:46.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:46.718+0000 7f18a82ea640 1 -- 172.21.15.32:0/1269696407 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18a019d840 con 0x7f18a01034c0 2024-05-03T18:47:46.725 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:46.719+0000 7f18977fe640 1 -- 172.21.15.32:0/1269696407 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f1898004fb0 con 0x7f18a01034c0 2024-05-03T18:47:46.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:46.720+0000 7f18977fe640 1 --2- 172.21.15.32:0/1269696407 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f188c0b2920 0x7f188c0b4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:46.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:46.720+0000 7f18a6086640 1 --2- 172.21.15.32:0/1269696407 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f188c0b2920 0x7f188c0b4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:46.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:46.720+0000 7f18977fe640 1 -- 172.21.15.32:0/1269696407 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8356+0+0 (secure 0 0 0) 0x7f18980e2a40 con 0x7f18a01034c0 2024-05-03T18:47:46.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:46.720+0000 7f18a6086640 1 --2- 172.21.15.32:0/1269696407 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f188c0b2920 0x7f188c0b4de0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f189c000f80 tx=0x7f189c005f30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:47:46.727 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:46.723+0000 7f18977fe640 1 -- 172.21.15.32:0/1269696407 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f1898054d70 con 0x7f18a01034c0 2024-05-03T18:47:46.959 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:46.958+0000 7f18a82ea640 1 -- 172.21.15.32:0/1269696407 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) -- 0x7f18a019da10 con 0x7f18a01034c0 2024-05-03T18:47:47.333 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:47.332+0000 7f18977fe640 1 -- 172.21.15.32:0/1269696407 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v26) ==== 118+0+0 (secure 0 0 0) 0x7f18980aa580 con 0x7f18a01034c0 2024-05-03T18:47:47.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:47.335+0000 7f18a82ea640 1 -- 172.21.15.32:0/1269696407 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f188c0b2920 msgr2=0x7f188c0b4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:47.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:47.335+0000 7f18a82ea640 1 --2- 172.21.15.32:0/1269696407 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f188c0b2920 0x7f188c0b4de0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f189c000f80 tx=0x7f189c005f30 comp rx=0 tx=0).stop 2024-05-03T18:47:47.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:47.335+0000 7f18a82ea640 1 -- 172.21.15.32:0/1269696407 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18a01034c0 msgr2=0x7f18a019bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:47.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:47.335+0000 7f18a82ea640 1 --2- 172.21.15.32:0/1269696407 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18a01034c0 0x7f18a019bd10 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f189800e970 tx=0x7f189800ed70 comp rx=0 tx=0).stop 2024-05-03T18:47:47.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:47.335+0000 7f18a82ea640 1 -- 172.21.15.32:0/1269696407 shutdown_connections 2024-05-03T18:47:47.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:47.335+0000 7f18a82ea640 1 --2- 172.21.15.32:0/1269696407 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f188c0b2920 0x7f188c0b4de0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:47.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:47.335+0000 7f18a82ea640 1 --2- 172.21.15.32:0/1269696407 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f18a0102b00 0x7f18a019b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:47.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:47.335+0000 7f18a82ea640 1 --2- 172.21.15.32:0/1269696407 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18a01034c0 0x7f18a019bd10 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:47.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:47.335+0000 7f18a82ea640 1 --2- 172.21.15.32:0/1269696407 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18a0101910 0x7f18a006ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:47.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:47.335+0000 7f18a82ea640 1 -- 172.21.15.32:0/1269696407 >> 172.21.15.32:0/1269696407 conn(0x7f18a00fce80 msgr2=0x7f18a0105e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:47:47.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:47.335+0000 7f18a82ea640 1 -- 172.21.15.32:0/1269696407 shutdown_connections 2024-05-03T18:47:47.337 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:47.335+0000 7f18a82ea640 1 -- 172.21.15.32:0/1269696407 wait complete. 2024-05-03T18:47:47.351 DEBUG:teuthology.orchestra.run.smithi032:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-05-03T18:47:47.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:47.458+0000 7f9a24dc8640 1 -- 172.21.15.32:0/3977117608 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a20101910 msgr2=0x7f9a20101d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:47.462 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:47.458+0000 7f9a24dc8640 1 --2- 172.21.15.32:0/3977117608 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a20101910 0x7f9a20101d30 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f9a10009ca0 tx=0x7f9a1002a8c0 comp rx=0 tx=0).stop 2024-05-03T18:47:47.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:47.459+0000 7f9a24dc8640 1 -- 172.21.15.32:0/3977117608 shutdown_connections 2024-05-03T18:47:47.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:47.459+0000 7f9a24dc8640 1 --2- 172.21.15.32:0/3977117608 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9a201034c0 0x7f9a20109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:47.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:47.459+0000 7f9a24dc8640 1 --2- 172.21.15.32:0/3977117608 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9a20102b00 0x7f9a20102f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:47.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:47.459+0000 7f9a24dc8640 1 --2- 172.21.15.32:0/3977117608 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a20101910 0x7f9a20101d30 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:47.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:47.459+0000 7f9a24dc8640 1 -- 172.21.15.32:0/3977117608 >> 172.21.15.32:0/3977117608 conn(0x7f9a200fce80 msgr2=0x7f9a200ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T18:47:47.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:47.459+0000 7f9a24dc8640 1 -- 172.21.15.32:0/3977117608 shutdown_connections 2024-05-03T18:47:47.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:47.459+0000 7f9a24dc8640 1 -- 172.21.15.32:0/3977117608 wait complete. 2024-05-03T18:47:47.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:47.459+0000 7f9a24dc8640 1 Processor -- start 2024-05-03T18:47:47.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:47.459+0000 7f9a24dc8640 1 -- start start 2024-05-03T18:47:47.463 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:47.460+0000 7f9a24dc8640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9a20101910 0x7f9a2019add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:47.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:47.460+0000 7f9a24dc8640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a20102b00 0x7f9a2019b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:47.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:47.460+0000 7f9a24dc8640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9a201034c0 0x7f9a201a2380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:47.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:47.460+0000 7f9a24dc8640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f9a2010d430 con 0x7f9a20102b00 2024-05-03T18:47:47.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:47.460+0000 7f9a24dc8640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f9a2010d2b0 con 0x7f9a20101910 2024-05-03T18:47:47.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:47.460+0000 7f9a24dc8640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f9a2010d5b0 con 0x7f9a201034c0 2024-05-03T18:47:47.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:47.460+0000 7f9a1dd9b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a20102b00 0x7f9a2019b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:47.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:47.460+0000 7f9a1dd9b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a20102b00 0x7f9a2019b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46602/0 (socket says 172.21.15.32:46602) 2024-05-03T18:47:47.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:47.460+0000 7f9a1dd9b640 1 -- 172.21.15.32:0/3388640237 learned_addr learned my addr 172.21.15.32:0/3388640237 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T18:47:47.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:47.460+0000 7f9a1e59c640 1 --2- 172.21.15.32:0/3388640237 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9a20101910 0x7f9a2019add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:47.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:47.460+0000 7f9a1dd9b640 1 -- 172.21.15.32:0/3388640237 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9a201034c0 msgr2=0x7f9a201a2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:47.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:47.460+0000 7f9a1ed9d640 1 --2- 172.21.15.32:0/3388640237 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9a201034c0 0x7f9a201a2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:47.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:47.460+0000 7f9a1dd9b640 1 --2- 172.21.15.32:0/3388640237 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9a201034c0 0x7f9a201a2380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:47.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:47.460+0000 7f9a1dd9b640 1 -- 172.21.15.32:0/3388640237 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9a20101910 msgr2=0x7f9a2019add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T18:47:47.464 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:47.460+0000 7f9a1dd9b640 1 --2- 172.21.15.32:0/3388640237 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9a20101910 0x7f9a2019add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T18:47:47.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:47.460+0000 7f9a1dd9b640 1 -- 172.21.15.32:0/3388640237 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a201a29f0 con 0x7f9a20102b00 2024-05-03T18:47:47.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:47.460+0000 7f9a1e59c640 1 --2- 172.21.15.32:0/3388640237 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9a20101910 0x7f9a2019add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T18:47:47.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:47.460+0000 7f9a1dd9b640 1 --2- 172.21.15.32:0/3388640237 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a20102b00 0x7f9a2019b310 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f9a1400e9f0 tx=0x7f9a1400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:47:47.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:47.460+0000 7f9a0f7fe640 1 -- 172.21.15.32:0/3388640237 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a1400cd80 con 0x7f9a20102b00 2024-05-03T18:47:47.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:47.460+0000 7f9a0f7fe640 1 -- 172.21.15.32:0/3388640237 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a140054d0 con 0x7f9a20102b00 2024-05-03T18:47:47.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:47.460+0000 7f9a0f7fe640 1 -- 172.21.15.32:0/3388640237 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a14010690 con 0x7f9a20102b00 2024-05-03T18:47:47.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:47.460+0000 7f9a24dc8640 1 -- 172.21.15.32:0/3388640237 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a201a2ce0 con 0x7f9a20102b00 2024-05-03T18:47:47.465 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:47.461+0000 7f9a24dc8640 1 -- 172.21.15.32:0/3388640237 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a201a3180 con 0x7f9a20102b00 2024-05-03T18:47:47.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:47.462+0000 7f9a24dc8640 1 -- 172.21.15.32:0/3388640237 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a201a3410 con 0x7f9a20102b00 2024-05-03T18:47:47.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:47.463+0000 7f9a0f7fe640 1 -- 172.21.15.32:0/3388640237 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f9a14004e90 con 0x7f9a20102b00 2024-05-03T18:47:47.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:47.463+0000 7f9a0f7fe640 1 --2- 172.21.15.32:0/3388640237 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f9a080b2920 0x7f9a080b4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T18:47:47.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:47.464+0000 7f9a0f7fe640 1 -- 172.21.15.32:0/3388640237 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8356+0+0 (secure 0 0 0) 0x7f9a140e2970 con 0x7f9a20102b00 2024-05-03T18:47:47.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:47.464+0000 7f9a1e59c640 1 --2- 172.21.15.32:0/3388640237 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f9a080b2920 0x7f9a080b4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T18:47:47.466 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:47.464+0000 7f9a1e59c640 1 --2- 172.21.15.32:0/3388640237 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f9a080b2920 0x7f9a080b4de0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f9a10000f80 tx=0x7f9a10005ef0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T18:47:47.469 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:47.468+0000 7f9a0f7fe640 1 -- 172.21.15.32:0/3388640237 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f9a140aa4b0 con 0x7f9a20102b00 2024-05-03T18:47:47.690 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:47:47.689+0000 7f9a24dc8640 1 -- 172.21.15.32:0/3388640237 --> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7f9a200ff050 con 0x7f9a080b2920 2024-05-03T18:47:47.747 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:47:47 smithi045 ceph-mon[33299]: pgmap v372: 97 pgs: 97 active+clean; 631 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:47:47.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:47:47 smithi045 ceph-mon[33299]: from='client.? 172.21.15.32:0/1269696407' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-05-03T18:47:47.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:47:47 smithi113 ceph-mon[33540]: pgmap v372: 97 pgs: 97 active+clean; 631 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:47:47.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:47:47 smithi113 ceph-mon[33540]: from='client.? 172.21.15.32:0/1269696407' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-05-03T18:47:47.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:47:47 smithi032 ceph-mon[29763]: pgmap v372: 97 pgs: 97 active+clean; 631 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:47:47.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:47:47 smithi032 ceph-mon[29763]: from='client.? 172.21.15.32:0/1269696407' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-05-03T18:47:48.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:47:48 smithi045 ceph-mon[33299]: from='client.? 172.21.15.32:0/1269696407' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-05-03T18:47:48.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:47:48 smithi045 ceph-mon[33299]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2024-05-03T18:47:48.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:47:48 smithi113 ceph-mon[33540]: from='client.? 172.21.15.32:0/1269696407' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-05-03T18:47:48.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:47:48 smithi113 ceph-mon[33540]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2024-05-03T18:47:48.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:47:48 smithi032 ceph-mon[29763]: from='client.? 172.21.15.32:0/1269696407' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-05-03T18:47:48.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:47:48 smithi032 ceph-mon[29763]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=f=up:active} 3 up:standby-replay 6 up:standby 2024-05-03T18:47:49.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:47:49 smithi045 ceph-mon[33299]: from='client.15150 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:47:49.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:47:49 smithi045 ceph-mon[33299]: pgmap v373: 97 pgs: 97 active+clean; 632 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 170 B/s wr, 6 op/s 2024-05-03T18:47:49.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:47:49 smithi113 ceph-mon[33540]: from='client.15150 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:47:49.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:47:49 smithi113 ceph-mon[33540]: pgmap v373: 97 pgs: 97 active+clean; 632 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 170 B/s wr, 6 op/s 2024-05-03T18:47:49.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:47:49 smithi032 ceph-mon[29763]: from='client.15150 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T18:47:49.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:47:49 smithi032 ceph-mon[29763]: pgmap v373: 97 pgs: 97 active+clean; 632 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 170 B/s wr, 6 op/s 2024-05-03T18:47:51.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:47:51 smithi045 ceph-mon[33299]: pgmap v374: 97 pgs: 97 active+clean; 632 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 170 B/s wr, 5 op/s 2024-05-03T18:47:51.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:47:51 smithi113 ceph-mon[33540]: pgmap v374: 97 pgs: 97 active+clean; 632 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 170 B/s wr, 5 op/s 2024-05-03T18:47:51.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:47:51 smithi032 ceph-mon[29763]: pgmap v374: 97 pgs: 97 active+clean; 632 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 170 B/s wr, 5 op/s 2024-05-03T18:47:53.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:47:53 smithi045 ceph-mon[33299]: pgmap v375: 97 pgs: 97 active+clean; 632 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 170 B/s wr, 6 op/s 2024-05-03T18:47:53.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:47:53 smithi113 ceph-mon[33540]: pgmap v375: 97 pgs: 97 active+clean; 632 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 170 B/s wr, 6 op/s 2024-05-03T18:47:53.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:47:53 smithi032 ceph-mon[29763]: pgmap v375: 97 pgs: 97 active+clean; 632 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 170 B/s wr, 6 op/s 2024-05-03T18:47:55.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:47:55 smithi032 ceph-mon[29763]: pgmap v376: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 682 B/s wr, 7 op/s 2024-05-03T18:47:55.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:47:55 smithi045 ceph-mon[33299]: pgmap v376: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 682 B/s wr, 7 op/s 2024-05-03T18:47:55.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:47:55 smithi113 ceph-mon[33540]: pgmap v376: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 682 B/s wr, 7 op/s 2024-05-03T18:47:57.747 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:47:57 smithi045 ceph-mon[33299]: pgmap v377: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 682 B/s wr, 6 op/s 2024-05-03T18:47:57.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:47:57 smithi113 ceph-mon[33540]: pgmap v377: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 682 B/s wr, 6 op/s 2024-05-03T18:47:57.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:47:57 smithi032 ceph-mon[29763]: pgmap v377: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 682 B/s wr, 6 op/s 2024-05-03T18:47:59.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:47:59 smithi045 ceph-mon[33299]: pgmap v378: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 682 B/s wr, 7 op/s 2024-05-03T18:47:59.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:47:59 smithi113 ceph-mon[33540]: pgmap v378: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 682 B/s wr, 7 op/s 2024-05-03T18:47:59.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:47:59 smithi032 ceph-mon[29763]: pgmap v378: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 682 B/s wr, 7 op/s 2024-05-03T18:48:01.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:48:01 smithi045 ceph-mon[33299]: pgmap v379: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 511 B/s wr, 5 op/s 2024-05-03T18:48:01.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:48:01 smithi113 ceph-mon[33540]: pgmap v379: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 511 B/s wr, 5 op/s 2024-05-03T18:48:01.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:48:01 smithi032 ceph-mon[29763]: pgmap v379: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 511 B/s wr, 5 op/s 2024-05-03T18:48:03.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:48:03 smithi045 ceph-mon[33299]: pgmap v380: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 511 B/s wr, 5 op/s 2024-05-03T18:48:03.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:48:03 smithi113 ceph-mon[33540]: pgmap v380: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 511 B/s wr, 5 op/s 2024-05-03T18:48:03.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:48:03 smithi032 ceph-mon[29763]: pgmap v380: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 511 B/s wr, 5 op/s 2024-05-03T18:48:04.747 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:48:04 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:48:04.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:48:04 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:48:04.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:48:04 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:48:05.719 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:48:05 smithi032 ceph-mon[29763]: pgmap v381: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 511 B/s wr, 6 op/s 2024-05-03T18:48:05.719 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:48:05 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:48:05.720 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:48:05 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:48:05.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:48:05 smithi045 ceph-mon[33299]: pgmap v381: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 511 B/s wr, 6 op/s 2024-05-03T18:48:05.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:48:05 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:48:05.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:48:05 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:48:05.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:48:05 smithi113 ceph-mon[33540]: pgmap v381: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 511 B/s wr, 6 op/s 2024-05-03T18:48:05.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:48:05 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:48:05.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:48:05 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:48:06.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:48:06 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:48:06.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:48:06 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:48:06.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:48:06 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:48:06.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:48:06 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:48:06.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:48:06 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:48:06.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:48:06 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:48:06.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:48:06 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:48:06.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:48:06 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:48:06.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:48:06 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:48:07.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:48:07 smithi045 ceph-mon[33299]: pgmap v382: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:48:08.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:48:07 smithi113 ceph-mon[33540]: pgmap v382: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:48:08.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:48:07 smithi032 ceph-mon[29763]: pgmap v382: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:48:09.997 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:48:09 smithi045 ceph-mon[33299]: pgmap v383: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:48:10.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:48:09 smithi113 ceph-mon[33540]: pgmap v383: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:48:10.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:48:09 smithi032 ceph-mon[29763]: pgmap v383: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:48:11.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:48:11 smithi045 ceph-mon[33299]: pgmap v384: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:48:12.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:48:11 smithi113 ceph-mon[33540]: pgmap v384: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:48:12.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:48:11 smithi032 ceph-mon[29763]: pgmap v384: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:48:13.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:48:13 smithi045 ceph-mon[33299]: pgmap v385: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:48:14.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:48:13 smithi113 ceph-mon[33540]: pgmap v385: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:48:14.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:48:13 smithi032 ceph-mon[29763]: pgmap v385: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:48:15.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:48:15 smithi045 ceph-mon[33299]: pgmap v386: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:48:16.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:48:15 smithi113 ceph-mon[33540]: pgmap v386: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:48:16.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:48:15 smithi032 ceph-mon[29763]: pgmap v386: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:48:17.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:48:17 smithi045 ceph-mon[33299]: pgmap v387: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:48:18.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:48:17 smithi113 ceph-mon[33540]: pgmap v387: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:48:18.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:48:17 smithi032 ceph-mon[29763]: pgmap v387: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:48:18.997 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:48:18 smithi045 ceph-mon[33299]: pgmap v388: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:48:19.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:48:18 smithi113 ceph-mon[33540]: pgmap v388: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:48:19.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:48:18 smithi032 ceph-mon[29763]: pgmap v388: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:48:21.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:48:21 smithi045 ceph-mon[33299]: pgmap v389: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:48:21.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:48:21 smithi113 ceph-mon[33540]: pgmap v389: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:48:21.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:48:21 smithi032 ceph-mon[29763]: pgmap v389: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:48:23.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:48:23 smithi045 ceph-mon[33299]: pgmap v390: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:48:23.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:48:23 smithi113 ceph-mon[33540]: pgmap v390: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:48:23.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:48:23 smithi032 ceph-mon[29763]: pgmap v390: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:48:25.353 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:48:25 smithi113 ceph-mon[33540]: pgmap v391: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:48:25.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:48:25 smithi045 ceph-mon[33299]: pgmap v391: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:48:25.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:48:25 smithi032 ceph-mon[29763]: pgmap v391: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:48:27.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:48:27 smithi045 ceph-mon[33299]: pgmap v392: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:48:27.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:48:27 smithi113 ceph-mon[33540]: pgmap v392: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:48:27.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:48:27 smithi032 ceph-mon[29763]: pgmap v392: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:48:29.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:48:29 smithi045 ceph-mon[33299]: pgmap v393: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:48:29.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:48:29 smithi113 ceph-mon[33540]: pgmap v393: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:48:29.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:48:29 smithi032 ceph-mon[29763]: pgmap v393: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:48:31.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:48:31 smithi045 ceph-mon[33299]: pgmap v394: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:48:31.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:48:31 smithi113 ceph-mon[33540]: pgmap v394: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:48:31.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:48:31 smithi032 ceph-mon[29763]: pgmap v394: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:48:33.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:48:33 smithi045 ceph-mon[33299]: pgmap v395: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:48:33.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:48:33 smithi113 ceph-mon[33540]: pgmap v395: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:48:33.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:48:33 smithi032 ceph-mon[29763]: pgmap v395: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:48:35.371 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:48:35 smithi032 ceph-mon[29763]: pgmap v396: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:48:35.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:48:35 smithi045 ceph-mon[33299]: pgmap v396: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:48:35.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:48:35 smithi113 ceph-mon[33540]: pgmap v396: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:48:37.497 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:48:37 smithi045 ceph-mon[33299]: pgmap v397: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:48:37.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:48:37 smithi113 ceph-mon[33540]: pgmap v397: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:48:37.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:48:37 smithi032 ceph-mon[29763]: pgmap v397: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:48:39.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:48:39 smithi045 ceph-mon[33299]: pgmap v398: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:48:39.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:48:39 smithi113 ceph-mon[33540]: pgmap v398: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:48:39.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:48:39 smithi032 ceph-mon[29763]: pgmap v398: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:48:41.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:48:41 smithi045 ceph-mon[33299]: pgmap v399: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:48:41.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:48:41 smithi113 ceph-mon[33540]: pgmap v399: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:48:41.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:48:41 smithi032 ceph-mon[29763]: pgmap v399: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:48:43.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:48:43 smithi045 ceph-mon[33299]: pgmap v400: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:48:43.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:48:43 smithi113 ceph-mon[33540]: pgmap v400: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:48:43.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:48:43 smithi032 ceph-mon[29763]: pgmap v400: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:48:45.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:48:44 smithi032 ceph-mon[29763]: pgmap v401: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:48:45.248 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:48:44 smithi045 ceph-mon[33299]: pgmap v401: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:48:45.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:48:44 smithi113 ceph-mon[33540]: pgmap v401: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:48:47.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:48:47 smithi045 ceph-mon[33299]: pgmap v402: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:48:47.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:48:47 smithi113 ceph-mon[33540]: pgmap v402: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:48:47.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:48:47 smithi032 ceph-mon[29763]: pgmap v402: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:48:49.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:48:49 smithi045 ceph-mon[33299]: pgmap v403: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:48:49.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:48:49 smithi113 ceph-mon[33540]: pgmap v403: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:48:49.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:48:49 smithi032 ceph-mon[29763]: pgmap v403: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:48:51.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:48:51 smithi045 ceph-mon[33299]: pgmap v404: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:48:51.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:48:51 smithi113 ceph-mon[33540]: pgmap v404: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:48:51.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:48:51 smithi032 ceph-mon[29763]: pgmap v404: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:48:53.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:48:53 smithi045 ceph-mon[33299]: pgmap v405: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:48:53.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:48:53 smithi113 ceph-mon[33540]: pgmap v405: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:48:53.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:48:53 smithi032 ceph-mon[29763]: pgmap v405: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:48:55.353 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:48:55 smithi113 ceph-mon[33540]: pgmap v406: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:48:55.497 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:48:55 smithi045 ceph-mon[33299]: pgmap v406: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:48:55.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:48:55 smithi032 ceph-mon[29763]: pgmap v406: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:48:57.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:48:57 smithi045 ceph-mon[33299]: pgmap v407: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:48:57.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:48:57 smithi113 ceph-mon[33540]: pgmap v407: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:48:57.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:48:57 smithi032 ceph-mon[29763]: pgmap v407: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:48:59.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:48:59 smithi045 ceph-mon[33299]: pgmap v408: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:48:59.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:48:59 smithi113 ceph-mon[33540]: pgmap v408: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:48:59.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:48:59 smithi032 ceph-mon[29763]: pgmap v408: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:49:01.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:49:01 smithi045 ceph-mon[33299]: pgmap v409: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:49:01.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:49:01 smithi113 ceph-mon[33540]: pgmap v409: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:49:01.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:49:01 smithi032 ceph-mon[29763]: pgmap v409: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:49:03.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:49:03 smithi045 ceph-mon[33299]: pgmap v410: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:49:03.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:49:03 smithi113 ceph-mon[33540]: pgmap v410: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:49:03.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:49:03 smithi032 ceph-mon[29763]: pgmap v410: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:49:05.370 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:49:05 smithi032 ceph-mon[29763]: pgmap v411: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:49:05.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:49:05 smithi045 ceph-mon[33299]: pgmap v411: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:49:05.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:49:05 smithi113 ceph-mon[33540]: pgmap v411: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:49:06.497 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:49:06 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:49:06.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:49:06 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:49:06.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:49:06 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:49:07.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:49:07 smithi032 ceph-mon[29763]: pgmap v412: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:49:07.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:49:07 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:49:07.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:49:07 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:49:07.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:49:07 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:49:07.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:49:07 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:49:07.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:49:07 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:49:07.569 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:49:07 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:49:07.569 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:49:07 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:49:07.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:49:07 smithi045 ceph-mon[33299]: pgmap v412: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:49:07.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:49:07 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:49:07.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:49:07 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:49:07.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:49:07 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:49:07.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:49:07 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:49:07.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:49:07 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:49:07.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:49:07 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:49:07.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:49:07 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:49:07.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:49:07 smithi113 ceph-mon[33540]: pgmap v412: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:49:07.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:49:07 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:49:07.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:49:07 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:49:07.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:49:07 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:49:07.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:49:07 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:49:07.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:49:07 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:49:07.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:49:07 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:49:07.802 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:49:07 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:49:09.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:49:09 smithi032 ceph-mon[29763]: pgmap v413: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:49:09.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:49:09 smithi045 ceph-mon[33299]: pgmap v413: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:49:09.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:49:09 smithi113 ceph-mon[33540]: pgmap v413: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:49:11.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:49:11 smithi032 ceph-mon[29763]: pgmap v414: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:49:11.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:49:11 smithi045 ceph-mon[33299]: pgmap v414: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:49:11.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:49:11 smithi113 ceph-mon[33540]: pgmap v414: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:49:13.747 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:49:13 smithi045 ceph-mon[33299]: pgmap v415: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:49:13.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:49:13 smithi113 ceph-mon[33540]: pgmap v415: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:49:13.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:49:13 smithi032 ceph-mon[29763]: pgmap v415: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:49:15.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:49:15 smithi045 ceph-mon[33299]: pgmap v416: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:49:15.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:49:15 smithi113 ceph-mon[33540]: pgmap v416: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:49:15.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:49:15 smithi032 ceph-mon[29763]: pgmap v416: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:49:17.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:49:17 smithi045 ceph-mon[33299]: pgmap v417: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:49:17.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:49:17 smithi113 ceph-mon[33540]: pgmap v417: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:49:17.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:49:17 smithi032 ceph-mon[29763]: pgmap v417: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:49:19.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:49:19 smithi045 ceph-mon[33299]: pgmap v418: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:49:19.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:49:19 smithi113 ceph-mon[33540]: pgmap v418: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:49:19.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:49:19 smithi032 ceph-mon[29763]: pgmap v418: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:49:21.747 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:49:21 smithi045 ceph-mon[33299]: pgmap v419: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:49:21.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:49:21 smithi113 ceph-mon[33540]: pgmap v419: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:49:21.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:49:21 smithi032 ceph-mon[29763]: pgmap v419: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:49:23.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:49:23 smithi045 ceph-mon[33299]: pgmap v420: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:49:23.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:49:23 smithi113 ceph-mon[33540]: pgmap v420: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:49:23.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:49:23 smithi032 ceph-mon[29763]: pgmap v420: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:49:25.603 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:49:25 smithi113 ceph-mon[33540]: pgmap v421: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:49:25.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:49:25 smithi045 ceph-mon[33299]: pgmap v421: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:49:25.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:49:25 smithi032 ceph-mon[29763]: pgmap v421: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:49:27.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:49:27 smithi045 ceph-mon[33299]: pgmap v422: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:49:27.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:49:27 smithi113 ceph-mon[33540]: pgmap v422: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:49:27.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:49:27 smithi032 ceph-mon[29763]: pgmap v422: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:49:29.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:49:29 smithi045 ceph-mon[33299]: pgmap v423: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:49:29.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:49:29 smithi113 ceph-mon[33540]: pgmap v423: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:49:29.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:49:29 smithi032 ceph-mon[29763]: pgmap v423: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:49:31.747 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:49:31 smithi045 ceph-mon[33299]: pgmap v424: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:49:31.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:49:31 smithi113 ceph-mon[33540]: pgmap v424: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:49:31.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:49:31 smithi032 ceph-mon[29763]: pgmap v424: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:49:33.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:49:33 smithi045 ceph-mon[33299]: pgmap v425: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:49:33.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:49:33 smithi113 ceph-mon[33540]: pgmap v425: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:49:33.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:49:33 smithi032 ceph-mon[29763]: pgmap v425: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:49:35.621 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:49:35 smithi032 ceph-mon[29763]: pgmap v426: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:49:35.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:49:35 smithi045 ceph-mon[33299]: pgmap v426: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:49:35.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:49:35 smithi113 ceph-mon[33540]: pgmap v426: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:49:37.747 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:49:37 smithi045 ceph-mon[33299]: pgmap v427: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:49:37.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:49:37 smithi113 ceph-mon[33540]: pgmap v427: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:49:37.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:49:37 smithi032 ceph-mon[29763]: pgmap v427: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:49:39.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:49:39 smithi045 ceph-mon[33299]: pgmap v428: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:49:39.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:49:39 smithi113 ceph-mon[33540]: pgmap v428: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:49:39.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:49:39 smithi032 ceph-mon[29763]: pgmap v428: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:49:41.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:49:41 smithi045 ceph-mon[33299]: pgmap v429: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:49:41.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:49:41 smithi113 ceph-mon[33540]: pgmap v429: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:49:41.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:49:41 smithi032 ceph-mon[29763]: pgmap v429: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:49:43.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:49:43 smithi045 ceph-mon[33299]: pgmap v430: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:49:43.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:49:43 smithi113 ceph-mon[33540]: pgmap v430: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:49:43.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:49:43 smithi032 ceph-mon[29763]: pgmap v430: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:49:45.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:49:45 smithi045 ceph-mon[33299]: pgmap v431: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:49:45.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:49:45 smithi113 ceph-mon[33540]: pgmap v431: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:49:45.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:49:45 smithi032 ceph-mon[29763]: pgmap v431: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:49:47.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:49:47 smithi045 ceph-mon[33299]: pgmap v432: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:49:47.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:49:47 smithi113 ceph-mon[33540]: pgmap v432: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:49:47.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:49:47 smithi032 ceph-mon[29763]: pgmap v432: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:49:49.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:49:49 smithi045 ceph-mon[33299]: pgmap v433: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:49:49.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:49:49 smithi113 ceph-mon[33540]: pgmap v433: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:49:49.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:49:49 smithi032 ceph-mon[29763]: pgmap v433: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:49:51.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:49:51 smithi045 ceph-mon[33299]: pgmap v434: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:49:51.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:49:51 smithi113 ceph-mon[33540]: pgmap v434: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:49:51.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:49:51 smithi032 ceph-mon[29763]: pgmap v434: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:49:53.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:49:53 smithi045 ceph-mon[33299]: pgmap v435: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:49:53.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:49:53 smithi113 ceph-mon[33540]: pgmap v435: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:49:53.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:49:53 smithi032 ceph-mon[29763]: pgmap v435: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:49:55.747 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:49:55 smithi045 ceph-mon[33299]: pgmap v436: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:49:55.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:49:55 smithi113 ceph-mon[33540]: pgmap v436: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:49:55.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:49:55 smithi032 ceph-mon[29763]: pgmap v436: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:49:57.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:49:57 smithi045 ceph-mon[33299]: pgmap v437: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:49:57.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:49:57 smithi113 ceph-mon[33540]: pgmap v437: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:49:57.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:49:57 smithi032 ceph-mon[29763]: pgmap v437: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:49:59.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:49:59 smithi045 ceph-mon[33299]: pgmap v438: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:49:59.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:49:59 smithi113 ceph-mon[33540]: pgmap v438: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:49:59.819 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:49:59 smithi032 ceph-mon[29763]: pgmap v438: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:50:00.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:50:00 smithi045 ceph-mon[33299]: overall HEALTH_OK 2024-05-03T18:50:00.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:50:00 smithi113 ceph-mon[33540]: overall HEALTH_OK 2024-05-03T18:50:00.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:50:00 smithi032 ceph-mon[29763]: overall HEALTH_OK 2024-05-03T18:50:01.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:50:01 smithi045 ceph-mon[33299]: pgmap v439: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:50:01.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:50:01 smithi113 ceph-mon[33540]: pgmap v439: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:50:01.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:50:01 smithi032 ceph-mon[29763]: pgmap v439: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:50:03.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:50:03 smithi045 ceph-mon[33299]: pgmap v440: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:50:03.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:50:03 smithi113 ceph-mon[33540]: pgmap v440: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:50:03.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:50:03 smithi032 ceph-mon[29763]: pgmap v440: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:50:05.747 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:50:05 smithi045 ceph-mon[33299]: pgmap v441: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:50:05.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:50:05 smithi113 ceph-mon[33540]: pgmap v441: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:50:05.820 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:50:05 smithi032 ceph-mon[29763]: pgmap v441: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:50:07.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:50:07 smithi045 ceph-mon[33299]: pgmap v442: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:50:07.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:50:07 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:50:07.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:50:07 smithi113 ceph-mon[33540]: pgmap v442: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:50:07.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:50:07 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:50:07.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:50:07 smithi032 ceph-mon[29763]: pgmap v442: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:50:07.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:50:07 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:50:08.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:50:08 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:50:08.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:50:08 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:50:08.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:50:08 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:50:08.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:50:08 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:50:08.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:50:08 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:50:08.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:50:08 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:50:08.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:50:08 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:50:08.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:50:08 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:50:08.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:50:08 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:50:09.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:50:09 smithi045 ceph-mon[33299]: pgmap v443: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:50:09.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:50:09 smithi113 ceph-mon[33540]: pgmap v443: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:50:09.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:50:09 smithi032 ceph-mon[29763]: pgmap v443: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:50:11.747 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:50:11 smithi045 ceph-mon[33299]: pgmap v444: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:50:11.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:50:11 smithi113 ceph-mon[33540]: pgmap v444: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:50:11.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:50:11 smithi032 ceph-mon[29763]: pgmap v444: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:50:13.747 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:50:13 smithi045 ceph-mon[33299]: pgmap v445: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:50:13.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:50:13 smithi113 ceph-mon[33540]: pgmap v445: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:50:13.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:50:13 smithi032 ceph-mon[29763]: pgmap v445: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:50:15.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:50:15 smithi045 ceph-mon[33299]: pgmap v446: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:50:15.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:50:15 smithi113 ceph-mon[33540]: pgmap v446: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:50:15.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:50:15 smithi032 ceph-mon[29763]: pgmap v446: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:50:17.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:50:17 smithi045 ceph-mon[33299]: pgmap v447: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:50:17.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:50:17 smithi113 ceph-mon[33540]: pgmap v447: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:50:17.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:50:17 smithi032 ceph-mon[29763]: pgmap v447: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:50:19.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:50:19 smithi045 ceph-mon[33299]: pgmap v448: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:50:19.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:50:19 smithi113 ceph-mon[33540]: pgmap v448: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:50:19.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:50:19 smithi032 ceph-mon[29763]: pgmap v448: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:50:21.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:50:21 smithi045 ceph-mon[33299]: pgmap v449: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:50:21.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:50:21 smithi113 ceph-mon[33540]: pgmap v449: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:50:21.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:50:21 smithi032 ceph-mon[29763]: pgmap v449: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:50:23.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:50:23 smithi045 ceph-mon[33299]: pgmap v450: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:50:23.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:50:23 smithi113 ceph-mon[33540]: pgmap v450: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:50:23.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:50:23 smithi032 ceph-mon[29763]: pgmap v450: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:50:25.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:50:25 smithi045 ceph-mon[33299]: pgmap v451: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:50:25.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:50:25 smithi113 ceph-mon[33540]: pgmap v451: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:50:25.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:50:25 smithi032 ceph-mon[29763]: pgmap v451: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:50:27.747 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:50:27 smithi045 ceph-mon[33299]: pgmap v452: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:50:27.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:50:27 smithi113 ceph-mon[33540]: pgmap v452: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:50:27.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:50:27 smithi032 ceph-mon[29763]: pgmap v452: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:50:29.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:50:29 smithi045 ceph-mon[33299]: pgmap v453: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:50:29.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:50:29 smithi113 ceph-mon[33540]: pgmap v453: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:50:29.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:50:29 smithi032 ceph-mon[29763]: pgmap v453: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:50:31.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:50:31 smithi045 ceph-mon[33299]: pgmap v454: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:50:31.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:50:31 smithi113 ceph-mon[33540]: pgmap v454: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:50:31.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:50:31 smithi032 ceph-mon[29763]: pgmap v454: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:50:33.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:50:33 smithi045 ceph-mon[33299]: pgmap v455: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:50:33.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:50:33 smithi113 ceph-mon[33540]: pgmap v455: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:50:33.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:50:33 smithi032 ceph-mon[29763]: pgmap v455: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:50:35.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:50:35 smithi045 ceph-mon[33299]: pgmap v456: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:50:35.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:50:35 smithi113 ceph-mon[33540]: pgmap v456: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:50:35.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:50:35 smithi032 ceph-mon[29763]: pgmap v456: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:50:37.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:50:37 smithi045 ceph-mon[33299]: pgmap v457: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:50:37.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:50:37 smithi113 ceph-mon[33540]: pgmap v457: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:50:37.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:50:37 smithi032 ceph-mon[29763]: pgmap v457: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:50:39.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:50:39 smithi045 ceph-mon[33299]: pgmap v458: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:50:39.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:50:39 smithi113 ceph-mon[33540]: pgmap v458: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:50:39.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:50:39 smithi032 ceph-mon[29763]: pgmap v458: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:50:41.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:50:41 smithi045 ceph-mon[33299]: pgmap v459: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:50:41.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:50:41 smithi113 ceph-mon[33540]: pgmap v459: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:50:41.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:50:41 smithi032 ceph-mon[29763]: pgmap v459: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:50:43.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:50:43 smithi045 ceph-mon[33299]: pgmap v460: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:50:43.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:50:43 smithi113 ceph-mon[33540]: pgmap v460: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:50:43.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:50:43 smithi032 ceph-mon[29763]: pgmap v460: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:50:45.747 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:50:45 smithi045 ceph-mon[33299]: pgmap v461: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:50:45.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:50:45 smithi113 ceph-mon[33540]: pgmap v461: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:50:45.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:50:45 smithi032 ceph-mon[29763]: pgmap v461: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:50:47.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:50:47.484+0000 7f9a0e7fc640 1 -- 172.21.15.32:0/3388640237 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7f9a040034c0 con 0x7f9a20102b00 2024-05-03T18:50:47.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:50:47.485+0000 7f9a0f7fe640 1 -- 172.21.15.32:0/3388640237 <== mon.0 v2:172.21.15.32:3300/0 7 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7f9a040034c0 con 0x7f9a20102b00 2024-05-03T18:50:47.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:50:47 smithi045 ceph-mon[33299]: pgmap v462: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:50:47.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:50:47 smithi113 ceph-mon[33540]: pgmap v462: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:50:47.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:50:47 smithi032 ceph-mon[29763]: pgmap v462: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:50:49.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:50:49 smithi045 ceph-mon[33299]: pgmap v463: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:50:49.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:50:49 smithi113 ceph-mon[33540]: pgmap v463: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:50:49.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:50:49 smithi032 ceph-mon[29763]: pgmap v463: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:50:51.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:50:51 smithi045 ceph-mon[33299]: pgmap v464: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:50:51.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:50:51 smithi113 ceph-mon[33540]: pgmap v464: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:50:51.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:50:51 smithi032 ceph-mon[29763]: pgmap v464: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:50:53.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:50:53 smithi045 ceph-mon[33299]: pgmap v465: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:50:53.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:50:53 smithi113 ceph-mon[33540]: pgmap v465: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:50:53.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:50:53 smithi032 ceph-mon[29763]: pgmap v465: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:50:55.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:50:55 smithi045 ceph-mon[33299]: pgmap v466: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:50:55.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:50:55 smithi113 ceph-mon[33540]: pgmap v466: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:50:55.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:50:55 smithi032 ceph-mon[29763]: pgmap v466: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:50:57.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:50:57 smithi045 ceph-mon[33299]: pgmap v467: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:50:57.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:50:57 smithi113 ceph-mon[33540]: pgmap v467: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:50:57.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:50:57 smithi032 ceph-mon[29763]: pgmap v467: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:50:59.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:50:59 smithi045 ceph-mon[33299]: pgmap v468: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:50:59.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:50:59 smithi113 ceph-mon[33540]: pgmap v468: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:50:59.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:50:59 smithi032 ceph-mon[29763]: pgmap v468: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:51:01.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:51:01 smithi045 ceph-mon[33299]: pgmap v469: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:51:01.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:51:01 smithi113 ceph-mon[33540]: pgmap v469: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:51:01.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:51:01 smithi032 ceph-mon[29763]: pgmap v469: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:51:03.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:51:03 smithi045 ceph-mon[33299]: pgmap v470: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:51:03.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:51:03 smithi113 ceph-mon[33540]: pgmap v470: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:51:03.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:51:03 smithi032 ceph-mon[29763]: pgmap v470: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:51:05.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:51:05 smithi045 ceph-mon[33299]: pgmap v471: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:51:05.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:51:05 smithi113 ceph-mon[33540]: pgmap v471: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:51:05.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:51:05 smithi032 ceph-mon[29763]: pgmap v471: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:51:07.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:51:07 smithi045 ceph-mon[33299]: pgmap v472: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:51:07.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:51:07 smithi113 ceph-mon[33540]: pgmap v472: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:51:07.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:51:07 smithi032 ceph-mon[29763]: pgmap v472: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:51:08.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:51:08 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:51:08.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:51:08 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:51:08.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:51:08 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:51:09.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:51:09 smithi113 ceph-mon[33540]: pgmap v473: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:51:09.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:51:09 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:51:09.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:51:09 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:51:09.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:51:09 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:51:09.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:51:09 smithi032 ceph-mon[29763]: pgmap v473: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:51:09.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:51:09 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:51:09.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:51:09 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:51:09.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:51:09 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:51:09.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:51:09 smithi045 ceph-mon[33299]: pgmap v473: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:51:09.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:51:09 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:51:09.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:51:09 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:51:09.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:51:09 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:51:11.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:51:11 smithi113 ceph-mon[33540]: pgmap v474: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:51:11.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:51:11 smithi032 ceph-mon[29763]: pgmap v474: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:51:11.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:51:11 smithi045 ceph-mon[33299]: pgmap v474: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:51:13.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:51:13 smithi113 ceph-mon[33540]: pgmap v475: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:51:13.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:51:13 smithi032 ceph-mon[29763]: pgmap v475: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:51:13.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:51:13 smithi045 ceph-mon[33299]: pgmap v475: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:51:15.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:51:15 smithi113 ceph-mon[33540]: pgmap v476: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:51:15.819 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:51:15 smithi032 ceph-mon[29763]: pgmap v476: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:51:15.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:51:15 smithi045 ceph-mon[33299]: pgmap v476: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:51:17.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:51:17 smithi113 ceph-mon[33540]: pgmap v477: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:51:17.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:51:17 smithi032 ceph-mon[29763]: pgmap v477: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:51:17.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:51:17 smithi045 ceph-mon[33299]: pgmap v477: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:51:19.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:51:19 smithi113 ceph-mon[33540]: pgmap v478: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:51:19.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:51:19 smithi032 ceph-mon[29763]: pgmap v478: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:51:19.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:51:19 smithi045 ceph-mon[33299]: pgmap v478: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:51:21.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:51:21 smithi113 ceph-mon[33540]: pgmap v479: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:51:21.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:51:21 smithi032 ceph-mon[29763]: pgmap v479: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:51:21.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:51:21 smithi045 ceph-mon[33299]: pgmap v479: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:51:23.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:51:23 smithi113 ceph-mon[33540]: pgmap v480: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:51:23.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:51:23 smithi032 ceph-mon[29763]: pgmap v480: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:51:23.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:51:23 smithi045 ceph-mon[33299]: pgmap v480: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:51:25.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:51:25 smithi113 ceph-mon[33540]: pgmap v481: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:51:25.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:51:25 smithi032 ceph-mon[29763]: pgmap v481: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:51:25.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:51:25 smithi045 ceph-mon[33299]: pgmap v481: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:51:27.800 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:51:27 smithi045 ceph-mon[33299]: pgmap v482: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:51:27.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:51:27 smithi113 ceph-mon[33540]: pgmap v482: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:51:27.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:51:27 smithi032 ceph-mon[29763]: pgmap v482: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:51:29.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:51:29 smithi113 ceph-mon[33540]: pgmap v483: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:51:29.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:51:29 smithi032 ceph-mon[29763]: pgmap v483: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:51:29.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:51:29 smithi045 ceph-mon[33299]: pgmap v483: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:51:31.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:51:31 smithi113 ceph-mon[33540]: pgmap v484: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:51:31.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:51:31 smithi032 ceph-mon[29763]: pgmap v484: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:51:31.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:51:31 smithi045 ceph-mon[33299]: pgmap v484: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:51:33.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:51:33 smithi113 ceph-mon[33540]: pgmap v485: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:51:33.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:51:33 smithi032 ceph-mon[29763]: pgmap v485: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:51:33.997 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:51:33 smithi045 ceph-mon[33299]: pgmap v485: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:51:35.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:51:35 smithi113 ceph-mon[33540]: pgmap v486: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:51:35.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:51:35 smithi032 ceph-mon[29763]: pgmap v486: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:51:35.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:51:35 smithi045 ceph-mon[33299]: pgmap v486: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:51:37.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:51:37 smithi113 ceph-mon[33540]: pgmap v487: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:51:37.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:51:37 smithi032 ceph-mon[29763]: pgmap v487: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:51:37.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:51:37 smithi045 ceph-mon[33299]: pgmap v487: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:51:39.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:51:39 smithi113 ceph-mon[33540]: pgmap v488: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:51:39.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:51:39 smithi032 ceph-mon[29763]: pgmap v488: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:51:39.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:51:39 smithi045 ceph-mon[33299]: pgmap v488: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:51:41.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:51:41 smithi113 ceph-mon[33540]: pgmap v489: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:51:41.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:51:41 smithi032 ceph-mon[29763]: pgmap v489: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:51:41.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:51:41 smithi045 ceph-mon[33299]: pgmap v489: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:51:43.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:51:43 smithi113 ceph-mon[33540]: pgmap v490: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:51:43.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:51:43 smithi032 ceph-mon[29763]: pgmap v490: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:51:43.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:51:43 smithi045 ceph-mon[33299]: pgmap v490: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:51:45.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:51:45 smithi032 ceph-mon[29763]: pgmap v491: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:51:45.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:51:45 smithi045 ceph-mon[33299]: pgmap v491: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:51:46.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:51:45 smithi113 ceph-mon[33540]: pgmap v491: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:51:47.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:51:47 smithi032 ceph-mon[29763]: pgmap v492: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:51:47.997 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:51:47 smithi045 ceph-mon[33299]: pgmap v492: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:51:48.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:51:47 smithi113 ceph-mon[33540]: pgmap v492: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:51:49.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:51:49 smithi032 ceph-mon[29763]: pgmap v493: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:51:49.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:51:49 smithi045 ceph-mon[33299]: pgmap v493: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:51:50.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:51:49 smithi113 ceph-mon[33540]: pgmap v493: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:51:51.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:51:51 smithi032 ceph-mon[29763]: pgmap v494: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:51:51.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:51:51 smithi045 ceph-mon[33299]: pgmap v494: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:51:52.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:51:51 smithi113 ceph-mon[33540]: pgmap v494: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:51:53.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:51:53 smithi032 ceph-mon[29763]: pgmap v495: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:51:53.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:51:53 smithi045 ceph-mon[33299]: pgmap v495: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:51:54.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:51:53 smithi113 ceph-mon[33540]: pgmap v495: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:51:55.819 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:51:55 smithi032 ceph-mon[29763]: pgmap v496: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:51:55.853 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:51:55 smithi113 ceph-mon[33540]: pgmap v496: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:51:55.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:51:55 smithi045 ceph-mon[33299]: pgmap v496: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:51:57.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:51:57 smithi045 ceph-mon[33299]: pgmap v497: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:51:58.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:51:57 smithi113 ceph-mon[33540]: pgmap v497: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:51:58.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:51:57 smithi032 ceph-mon[29763]: pgmap v497: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:51:59.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:51:59 smithi045 ceph-mon[33299]: pgmap v498: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:52:00.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:51:59 smithi113 ceph-mon[33540]: pgmap v498: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:52:00.070 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:51:59 smithi032 ceph-mon[29763]: pgmap v498: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:52:01.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:52:01 smithi045 ceph-mon[33299]: pgmap v499: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:52:02.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:52:01 smithi113 ceph-mon[33540]: pgmap v499: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:52:02.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:52:01 smithi032 ceph-mon[29763]: pgmap v499: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:52:03.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:52:03 smithi045 ceph-mon[33299]: pgmap v500: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:52:04.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:52:03 smithi113 ceph-mon[33540]: pgmap v500: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:52:04.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:52:03 smithi032 ceph-mon[29763]: pgmap v500: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:52:05.870 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:52:05 smithi032 ceph-mon[29763]: pgmap v501: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:52:05.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:52:05 smithi045 ceph-mon[33299]: pgmap v501: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:52:06.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:52:05 smithi113 ceph-mon[33540]: pgmap v501: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:52:07.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:52:07 smithi045 ceph-mon[33299]: pgmap v502: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:52:08.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:52:07 smithi113 ceph-mon[33540]: pgmap v502: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:52:08.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:52:07 smithi032 ceph-mon[29763]: pgmap v502: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:52:09.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:52:09 smithi045 ceph-mon[33299]: pgmap v503: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:52:09.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:52:09 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:52:09.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:52:09 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:52:09.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:52:09 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:52:09.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:52:09 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:52:10.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:52:09 smithi113 ceph-mon[33540]: pgmap v503: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:52:10.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:52:09 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:52:10.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:52:09 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:52:10.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:52:09 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:52:10.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:52:09 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:52:10.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:52:09 smithi032 ceph-mon[29763]: pgmap v503: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:52:10.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:52:09 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:52:10.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:52:09 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:52:10.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:52:09 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:52:10.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:52:09 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:52:11.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:52:11 smithi045 ceph-mon[33299]: pgmap v504: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:52:12.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:52:11 smithi113 ceph-mon[33540]: pgmap v504: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:52:12.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:52:11 smithi032 ceph-mon[29763]: pgmap v504: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:52:13.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:52:13 smithi045 ceph-mon[33299]: pgmap v505: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:52:14.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:52:13 smithi113 ceph-mon[33540]: pgmap v505: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:52:14.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:52:13 smithi032 ceph-mon[29763]: pgmap v505: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:52:15.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:52:15 smithi045 ceph-mon[33299]: pgmap v506: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:52:16.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:52:15 smithi113 ceph-mon[33540]: pgmap v506: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:52:16.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:52:15 smithi032 ceph-mon[29763]: pgmap v506: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:52:17.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:52:17 smithi045 ceph-mon[33299]: pgmap v507: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:52:18.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:52:17 smithi113 ceph-mon[33540]: pgmap v507: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:52:18.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:52:17 smithi032 ceph-mon[29763]: pgmap v507: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:52:19.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:52:19 smithi045 ceph-mon[33299]: pgmap v508: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:52:20.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:52:19 smithi113 ceph-mon[33540]: pgmap v508: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:52:20.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:52:19 smithi032 ceph-mon[29763]: pgmap v508: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:52:21.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:52:21 smithi045 ceph-mon[33299]: pgmap v509: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:52:22.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:52:21 smithi113 ceph-mon[33540]: pgmap v509: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:52:22.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:52:21 smithi032 ceph-mon[29763]: pgmap v509: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:52:23.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:52:23 smithi045 ceph-mon[33299]: pgmap v510: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:52:24.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:52:23 smithi113 ceph-mon[33540]: pgmap v510: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:52:24.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:52:23 smithi032 ceph-mon[29763]: pgmap v510: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:52:25.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:52:25 smithi045 ceph-mon[33299]: pgmap v511: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:52:26.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:52:25 smithi113 ceph-mon[33540]: pgmap v511: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:52:26.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:52:25 smithi032 ceph-mon[29763]: pgmap v511: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:52:27.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:52:27 smithi045 ceph-mon[33299]: pgmap v512: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:52:28.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:52:27 smithi113 ceph-mon[33540]: pgmap v512: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:52:28.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:52:27 smithi032 ceph-mon[29763]: pgmap v512: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:52:29.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:52:29 smithi045 ceph-mon[33299]: pgmap v513: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:52:30.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:52:29 smithi113 ceph-mon[33540]: pgmap v513: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:52:30.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:52:29 smithi032 ceph-mon[29763]: pgmap v513: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:52:31.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:52:31 smithi045 ceph-mon[33299]: pgmap v514: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:52:32.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:52:31 smithi113 ceph-mon[33540]: pgmap v514: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:52:32.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:52:31 smithi032 ceph-mon[29763]: pgmap v514: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:52:33.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:52:33 smithi045 ceph-mon[33299]: pgmap v515: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:52:34.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:52:33 smithi113 ceph-mon[33540]: pgmap v515: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:52:34.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:52:33 smithi032 ceph-mon[29763]: pgmap v515: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:52:35.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:52:35 smithi045 ceph-mon[33299]: pgmap v516: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:52:36.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:52:35 smithi113 ceph-mon[33540]: pgmap v516: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:52:36.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:52:35 smithi032 ceph-mon[29763]: pgmap v516: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:52:37.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:52:37 smithi045 ceph-mon[33299]: pgmap v517: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:52:38.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:52:37 smithi113 ceph-mon[33540]: pgmap v517: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:52:38.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:52:37 smithi032 ceph-mon[29763]: pgmap v517: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:52:39.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:52:39 smithi045 ceph-mon[33299]: pgmap v518: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:52:40.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:52:39 smithi113 ceph-mon[33540]: pgmap v518: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:52:40.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:52:39 smithi032 ceph-mon[29763]: pgmap v518: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:52:41.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:52:41 smithi045 ceph-mon[33299]: pgmap v519: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:52:42.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:52:41 smithi113 ceph-mon[33540]: pgmap v519: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:52:42.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:52:41 smithi032 ceph-mon[29763]: pgmap v519: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:52:43.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:52:43 smithi045 ceph-mon[33299]: pgmap v520: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:52:44.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:52:43 smithi113 ceph-mon[33540]: pgmap v520: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:52:44.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:52:43 smithi032 ceph-mon[29763]: pgmap v520: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:52:45.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:52:45 smithi045 ceph-mon[33299]: pgmap v521: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:52:46.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:52:45 smithi113 ceph-mon[33540]: pgmap v521: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:52:46.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:52:45 smithi032 ceph-mon[29763]: pgmap v521: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:52:47.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:52:47 smithi045 ceph-mon[33299]: pgmap v522: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:52:48.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:52:47 smithi113 ceph-mon[33540]: pgmap v522: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:52:48.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:52:47 smithi032 ceph-mon[29763]: pgmap v522: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:52:49.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:52:49 smithi045 ceph-mon[33299]: pgmap v523: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:52:50.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:52:49 smithi113 ceph-mon[33540]: pgmap v523: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:52:50.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:52:49 smithi032 ceph-mon[29763]: pgmap v523: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:52:51.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:52:51 smithi045 ceph-mon[33299]: pgmap v524: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:52:52.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:52:51 smithi113 ceph-mon[33540]: pgmap v524: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:52:52.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:52:51 smithi032 ceph-mon[29763]: pgmap v524: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:52:53.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:52:53 smithi045 ceph-mon[33299]: pgmap v525: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:52:54.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:52:53 smithi113 ceph-mon[33540]: pgmap v525: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:52:54.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:52:53 smithi032 ceph-mon[29763]: pgmap v525: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:52:55.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:52:55 smithi045 ceph-mon[33299]: pgmap v526: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:52:56.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:52:55 smithi113 ceph-mon[33540]: pgmap v526: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:52:56.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:52:55 smithi032 ceph-mon[29763]: pgmap v526: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:52:57.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:52:57 smithi045 ceph-mon[33299]: pgmap v527: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:52:58.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:52:57 smithi113 ceph-mon[33540]: pgmap v527: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:52:58.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:52:57 smithi032 ceph-mon[29763]: pgmap v527: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:52:58.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:52:58 smithi045 ceph-mon[33299]: pgmap v528: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:52:59.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:52:58 smithi113 ceph-mon[33540]: pgmap v528: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:52:59.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:52:58 smithi032 ceph-mon[29763]: pgmap v528: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:53:01.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:53:01 smithi045 ceph-mon[33299]: pgmap v529: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:53:01.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:53:01 smithi113 ceph-mon[33540]: pgmap v529: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:53:01.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:53:01 smithi032 ceph-mon[29763]: pgmap v529: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:53:03.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:53:03 smithi045 ceph-mon[33299]: pgmap v530: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:53:03.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:53:03 smithi113 ceph-mon[33540]: pgmap v530: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:53:03.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:53:03 smithi032 ceph-mon[29763]: pgmap v530: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:53:05.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:53:05 smithi045 ceph-mon[33299]: pgmap v531: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:53:05.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:53:05 smithi113 ceph-mon[33540]: pgmap v531: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:53:05.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:53:05 smithi032 ceph-mon[29763]: pgmap v531: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:53:07.497 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:53:07 smithi045 ceph-mon[33299]: pgmap v532: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:53:07.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:53:07 smithi113 ceph-mon[33540]: pgmap v532: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:53:07.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:53:07 smithi032 ceph-mon[29763]: pgmap v532: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:53:09.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:53:09 smithi045 ceph-mon[33299]: pgmap v533: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:53:09.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:53:09 smithi113 ceph-mon[33540]: pgmap v533: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:53:09.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:53:09 smithi032 ceph-mon[29763]: pgmap v533: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:53:10.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:53:10 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:53:10.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:53:10 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:53:10.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:53:10 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:53:11.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:53:11 smithi045 ceph-mon[33299]: pgmap v534: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:53:11.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:53:11 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:53:11.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:53:11 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:53:11.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:53:11 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:53:11.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:53:11 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:53:11.499 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:53:11 smithi045 ceph-mon[33299]: Adjusting osd_memory_target on smithi032 to 177.2M 2024-05-03T18:53:11.499 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:53:11 smithi045 ceph-mon[33299]: Unable to set osd_memory_target on smithi032 to 185897062: error parsing value: Value '185897062' is below minimum 939524096 2024-05-03T18:53:11.499 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:53:11 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:53:11.499 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:53:11 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:53:11.499 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:53:11 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:53:11.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:53:11 smithi113 ceph-mon[33540]: pgmap v534: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:53:11.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:53:11 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:53:11.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:53:11 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:53:11.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:53:11 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:53:11.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:53:11 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:53:11.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:53:11 smithi113 ceph-mon[33540]: Adjusting osd_memory_target on smithi032 to 177.2M 2024-05-03T18:53:11.552 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:53:11 smithi113 ceph-mon[33540]: Unable to set osd_memory_target on smithi032 to 185897062: error parsing value: Value '185897062' is below minimum 939524096 2024-05-03T18:53:11.552 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:53:11 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:53:11.552 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:53:11 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:53:11.552 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:53:11 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:53:11.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:53:11 smithi032 ceph-mon[29763]: pgmap v534: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:53:11.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:53:11 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:53:11.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:53:11 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:53:11.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:53:11 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:53:11.569 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:53:11 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:53:11.569 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:53:11 smithi032 ceph-mon[29763]: Adjusting osd_memory_target on smithi032 to 177.2M 2024-05-03T18:53:11.569 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:53:11 smithi032 ceph-mon[29763]: Unable to set osd_memory_target on smithi032 to 185897062: error parsing value: Value '185897062' is below minimum 939524096 2024-05-03T18:53:11.569 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:53:11 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:53:11.569 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:53:11 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:53:11.569 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:53:11 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:53:13.499 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:53:13 smithi045 ceph-mon[33299]: pgmap v535: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:53:13.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:53:13 smithi113 ceph-mon[33540]: pgmap v535: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:53:13.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:53:13 smithi032 ceph-mon[29763]: pgmap v535: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:53:15.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:53:15 smithi045 ceph-mon[33299]: pgmap v536: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:53:15.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:53:15 smithi113 ceph-mon[33540]: pgmap v536: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:53:15.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:53:15 smithi032 ceph-mon[29763]: pgmap v536: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:53:17.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:53:17 smithi045 ceph-mon[33299]: pgmap v537: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:53:17.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:53:17 smithi113 ceph-mon[33540]: pgmap v537: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:53:17.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:53:17 smithi032 ceph-mon[29763]: pgmap v537: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:53:19.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:53:19 smithi045 ceph-mon[33299]: pgmap v538: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:53:19.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:53:19 smithi113 ceph-mon[33540]: pgmap v538: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:53:19.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:53:19 smithi032 ceph-mon[29763]: pgmap v538: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:53:21.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:53:21 smithi045 ceph-mon[33299]: pgmap v539: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:53:21.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:53:21 smithi113 ceph-mon[33540]: pgmap v539: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:53:21.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:53:21 smithi032 ceph-mon[29763]: pgmap v539: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:53:23.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:53:23 smithi045 ceph-mon[33299]: pgmap v540: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:53:23.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:53:23 smithi113 ceph-mon[33540]: pgmap v540: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:53:23.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:53:23 smithi032 ceph-mon[29763]: pgmap v540: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:53:25.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:53:25 smithi045 ceph-mon[33299]: pgmap v541: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:53:25.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:53:25 smithi113 ceph-mon[33540]: pgmap v541: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:53:25.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:53:25 smithi032 ceph-mon[29763]: pgmap v541: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:53:27.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:53:27 smithi045 ceph-mon[33299]: pgmap v542: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:53:27.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:53:27 smithi113 ceph-mon[33540]: pgmap v542: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:53:27.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:53:27 smithi032 ceph-mon[29763]: pgmap v542: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:53:29.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:53:29 smithi045 ceph-mon[33299]: pgmap v543: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:53:29.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:53:29 smithi113 ceph-mon[33540]: pgmap v543: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:53:29.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:53:29 smithi032 ceph-mon[29763]: pgmap v543: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:53:31.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:53:31 smithi045 ceph-mon[33299]: pgmap v544: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:53:31.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:53:31 smithi113 ceph-mon[33540]: pgmap v544: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:53:31.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:53:31 smithi032 ceph-mon[29763]: pgmap v544: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:53:33.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:53:33 smithi045 ceph-mon[33299]: pgmap v545: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:53:33.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:53:33 smithi113 ceph-mon[33540]: pgmap v545: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:53:33.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:53:33 smithi032 ceph-mon[29763]: pgmap v545: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:53:35.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:53:35 smithi045 ceph-mon[33299]: pgmap v546: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:53:35.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:53:35 smithi113 ceph-mon[33540]: pgmap v546: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:53:35.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:53:35 smithi032 ceph-mon[29763]: pgmap v546: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:53:37.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:53:37 smithi045 ceph-mon[33299]: pgmap v547: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:53:37.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:53:37 smithi113 ceph-mon[33540]: pgmap v547: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:53:37.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:53:37 smithi032 ceph-mon[29763]: pgmap v547: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:53:39.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:53:39 smithi045 ceph-mon[33299]: pgmap v548: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:53:39.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:53:39 smithi113 ceph-mon[33540]: pgmap v548: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:53:39.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:53:39 smithi032 ceph-mon[29763]: pgmap v548: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:53:41.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:53:41 smithi045 ceph-mon[33299]: pgmap v549: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:53:41.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:53:41 smithi113 ceph-mon[33540]: pgmap v549: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:53:41.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:53:41 smithi032 ceph-mon[29763]: pgmap v549: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:53:43.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:53:43 smithi045 ceph-mon[33299]: pgmap v550: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:53:43.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:53:43 smithi113 ceph-mon[33540]: pgmap v550: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:53:43.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:53:43 smithi032 ceph-mon[29763]: pgmap v550: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:53:45.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:53:45 smithi045 ceph-mon[33299]: pgmap v551: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:53:45.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:53:45 smithi113 ceph-mon[33540]: pgmap v551: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:53:45.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:53:45 smithi032 ceph-mon[29763]: pgmap v551: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:53:47.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:53:47 smithi045 ceph-mon[33299]: pgmap v552: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:53:47.512 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:53:47.511+0000 7f9a0e7fc640 1 -- 172.21.15.32:0/3388640237 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7f9a04005850 con 0x7f9a20102b00 2024-05-03T18:53:47.513 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:53:47.511+0000 7f9a0f7fe640 1 -- 172.21.15.32:0/3388640237 <== mon.0 v2:172.21.15.32:3300/0 8 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7f9a04005850 con 0x7f9a20102b00 2024-05-03T18:53:47.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:53:47 smithi113 ceph-mon[33540]: pgmap v552: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:53:47.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:53:47 smithi032 ceph-mon[29763]: pgmap v552: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:53:49.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:53:49 smithi045 ceph-mon[33299]: pgmap v553: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:53:49.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:53:49 smithi113 ceph-mon[33540]: pgmap v553: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:53:49.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:53:49 smithi032 ceph-mon[29763]: pgmap v553: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:53:51.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:53:51 smithi045 ceph-mon[33299]: pgmap v554: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:53:51.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:53:51 smithi113 ceph-mon[33540]: pgmap v554: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:53:51.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:53:51 smithi032 ceph-mon[29763]: pgmap v554: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:53:53.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:53:53 smithi045 ceph-mon[33299]: pgmap v555: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:53:53.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:53:53 smithi113 ceph-mon[33540]: pgmap v555: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:53:53.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:53:53 smithi032 ceph-mon[29763]: pgmap v555: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:53:55.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:53:55 smithi045 ceph-mon[33299]: pgmap v556: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:53:55.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:53:55 smithi113 ceph-mon[33540]: pgmap v556: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:53:55.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:53:55 smithi032 ceph-mon[29763]: pgmap v556: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:53:57.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:53:57 smithi113 ceph-mon[33540]: pgmap v557: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:53:57.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:53:57 smithi032 ceph-mon[29763]: pgmap v557: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:53:57.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:53:57 smithi045 ceph-mon[33299]: pgmap v557: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:53:59.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:53:59 smithi113 ceph-mon[33540]: pgmap v558: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:53:59.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:53:59 smithi032 ceph-mon[29763]: pgmap v558: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:53:59.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:53:59 smithi045 ceph-mon[33299]: pgmap v558: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:54:01.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:54:01 smithi113 ceph-mon[33540]: pgmap v559: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:54:01.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:54:01 smithi032 ceph-mon[29763]: pgmap v559: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:54:01.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:54:01 smithi045 ceph-mon[33299]: pgmap v559: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:54:03.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:54:03 smithi113 ceph-mon[33540]: pgmap v560: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:54:03.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:54:03 smithi032 ceph-mon[29763]: pgmap v560: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:54:03.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:54:03 smithi045 ceph-mon[33299]: pgmap v560: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:54:05.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:54:05 smithi113 ceph-mon[33540]: pgmap v561: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:54:05.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:54:05 smithi032 ceph-mon[29763]: pgmap v561: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:54:05.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:54:05 smithi045 ceph-mon[33299]: pgmap v561: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:54:07.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:54:07 smithi113 ceph-mon[33540]: pgmap v562: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:54:07.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:54:07 smithi032 ceph-mon[29763]: pgmap v562: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:54:07.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:54:07 smithi045 ceph-mon[33299]: pgmap v562: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:54:09.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:54:09 smithi113 ceph-mon[33540]: pgmap v563: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:54:09.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:54:09 smithi032 ceph-mon[29763]: pgmap v563: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:54:09.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:54:09 smithi045 ceph-mon[33299]: pgmap v563: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:54:11.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:54:11 smithi113 ceph-mon[33540]: pgmap v564: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:54:11.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:54:11 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:54:11.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:54:11 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:54:11.552 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:54:11 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:54:11.552 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:54:11 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:54:11.552 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:54:11 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:54:11.552 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:54:11 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:54:11.552 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:54:11 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:54:11.552 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:54:11 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:54:11.552 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:54:11 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:54:11.552 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:54:11 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:54:11.552 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:54:11 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:54:11.552 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:54:11 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:54:11.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:54:11 smithi032 ceph-mon[29763]: pgmap v564: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:54:11.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:54:11 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:54:11.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:54:11 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:54:11.569 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:54:11 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:54:11.569 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:54:11 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:54:11.569 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:54:11 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:54:11.569 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:54:11 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:54:11.569 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:54:11 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:54:11.569 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:54:11 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:54:11.569 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:54:11 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:54:11.569 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:54:11 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:54:11.569 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:54:11 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:54:11.569 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:54:11 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:54:11.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:54:11 smithi045 ceph-mon[33299]: pgmap v564: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:54:11.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:54:11 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:54:11.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:54:11 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:54:11.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:54:11 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:54:11.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:54:11 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:54:11.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:54:11 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:54:11.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:54:11 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:54:11.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:54:11 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:54:11.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:54:11 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:54:11.749 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:54:11 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-03T18:54:11.749 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:54:11 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:54:11.749 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:54:11 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:54:11.749 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:54:11 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:54:12.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:54:12 smithi113 ceph-mon[33540]: Adjusting osd_memory_target on smithi045 to 177.2M 2024-05-03T18:54:12.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:54:12 smithi113 ceph-mon[33540]: Unable to set osd_memory_target on smithi045 to 185893478: error parsing value: Value '185893478' is below minimum 939524096 2024-05-03T18:54:12.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:54:12 smithi113 ceph-mon[33540]: Adjusting osd_memory_target on smithi113 to 176.2M 2024-05-03T18:54:12.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:54:12 smithi113 ceph-mon[33540]: Unable to set osd_memory_target on smithi113 to 184772403: error parsing value: Value '184772403' is below minimum 939524096 2024-05-03T18:54:12.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:54:12 smithi032 ceph-mon[29763]: Adjusting osd_memory_target on smithi045 to 177.2M 2024-05-03T18:54:12.569 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:54:12 smithi032 ceph-mon[29763]: Unable to set osd_memory_target on smithi045 to 185893478: error parsing value: Value '185893478' is below minimum 939524096 2024-05-03T18:54:12.569 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:54:12 smithi032 ceph-mon[29763]: Adjusting osd_memory_target on smithi113 to 176.2M 2024-05-03T18:54:12.569 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:54:12 smithi032 ceph-mon[29763]: Unable to set osd_memory_target on smithi113 to 184772403: error parsing value: Value '184772403' is below minimum 939524096 2024-05-03T18:54:12.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:54:12 smithi045 ceph-mon[33299]: Adjusting osd_memory_target on smithi045 to 177.2M 2024-05-03T18:54:12.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:54:12 smithi045 ceph-mon[33299]: Unable to set osd_memory_target on smithi045 to 185893478: error parsing value: Value '185893478' is below minimum 939524096 2024-05-03T18:54:12.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:54:12 smithi045 ceph-mon[33299]: Adjusting osd_memory_target on smithi113 to 176.2M 2024-05-03T18:54:12.749 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:54:12 smithi045 ceph-mon[33299]: Unable to set osd_memory_target on smithi113 to 184772403: error parsing value: Value '184772403' is below minimum 939524096 2024-05-03T18:54:13.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:54:13 smithi113 ceph-mon[33540]: pgmap v565: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:54:13.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:54:13 smithi032 ceph-mon[29763]: pgmap v565: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:54:13.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:54:13 smithi045 ceph-mon[33299]: pgmap v565: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:54:15.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:54:15 smithi113 ceph-mon[33540]: pgmap v566: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:54:15.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:54:15 smithi032 ceph-mon[29763]: pgmap v566: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:54:15.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:54:15 smithi045 ceph-mon[33299]: pgmap v566: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:54:17.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:54:17 smithi113 ceph-mon[33540]: pgmap v567: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:54:17.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:54:17 smithi032 ceph-mon[29763]: pgmap v567: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:54:17.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:54:17 smithi045 ceph-mon[33299]: pgmap v567: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:54:19.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:54:19 smithi113 ceph-mon[33540]: pgmap v568: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:54:19.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:54:19 smithi032 ceph-mon[29763]: pgmap v568: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:54:19.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:54:19 smithi045 ceph-mon[33299]: pgmap v568: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:54:21.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:54:21 smithi113 ceph-mon[33540]: pgmap v569: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:54:21.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:54:21 smithi032 ceph-mon[29763]: pgmap v569: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:54:21.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:54:21 smithi045 ceph-mon[33299]: pgmap v569: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:54:23.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:54:23 smithi113 ceph-mon[33540]: pgmap v570: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:54:23.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:54:23 smithi032 ceph-mon[29763]: pgmap v570: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:54:23.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:54:23 smithi045 ceph-mon[33299]: pgmap v570: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:54:25.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:54:25 smithi113 ceph-mon[33540]: pgmap v571: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:54:25.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:54:25 smithi032 ceph-mon[29763]: pgmap v571: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:54:25.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:54:25 smithi045 ceph-mon[33299]: pgmap v571: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:54:27.550 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:54:27 smithi045 ceph-mon[33299]: pgmap v572: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:54:27.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:54:27 smithi113 ceph-mon[33540]: pgmap v572: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:54:27.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:54:27 smithi032 ceph-mon[29763]: pgmap v572: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:54:29.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:54:29 smithi113 ceph-mon[33540]: pgmap v573: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:54:29.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:54:29 smithi032 ceph-mon[29763]: pgmap v573: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:54:29.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:54:29 smithi045 ceph-mon[33299]: pgmap v573: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:54:31.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:54:31 smithi032 ceph-mon[29763]: pgmap v574: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:54:31.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:54:31 smithi045 ceph-mon[33299]: pgmap v574: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:54:31.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:54:31 smithi113 ceph-mon[33540]: pgmap v574: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:54:33.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:54:33 smithi032 ceph-mon[29763]: pgmap v575: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:54:33.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:54:33 smithi045 ceph-mon[33299]: pgmap v575: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:54:33.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:54:33 smithi113 ceph-mon[33540]: pgmap v575: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:54:35.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:54:35 smithi032 ceph-mon[29763]: pgmap v576: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:54:35.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:54:35 smithi045 ceph-mon[33299]: pgmap v576: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:54:35.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:54:35 smithi113 ceph-mon[33540]: pgmap v576: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:54:37.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:54:37 smithi032 ceph-mon[29763]: pgmap v577: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:54:37.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:54:37 smithi045 ceph-mon[33299]: pgmap v577: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:54:37.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:54:37 smithi113 ceph-mon[33540]: pgmap v577: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:54:39.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:54:39 smithi032 ceph-mon[29763]: pgmap v578: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:54:39.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:54:39 smithi045 ceph-mon[33299]: pgmap v578: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:54:39.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:54:39 smithi113 ceph-mon[33540]: pgmap v578: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:54:41.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:54:41 smithi045 ceph-mon[33299]: pgmap v579: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:54:41.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:54:41 smithi113 ceph-mon[33540]: pgmap v579: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:54:41.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:54:41 smithi032 ceph-mon[29763]: pgmap v579: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:54:43.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:54:43 smithi045 ceph-mon[33299]: pgmap v580: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:54:43.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:54:43 smithi113 ceph-mon[33540]: pgmap v580: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:54:43.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:54:43 smithi032 ceph-mon[29763]: pgmap v580: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:54:45.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:54:45 smithi045 ceph-mon[33299]: pgmap v581: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:54:45.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:54:45 smithi113 ceph-mon[33540]: pgmap v581: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:54:45.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:54:45 smithi032 ceph-mon[29763]: pgmap v581: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:54:47.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:54:47 smithi045 ceph-mon[33299]: pgmap v582: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:54:47.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:54:47 smithi113 ceph-mon[33540]: pgmap v582: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:54:47.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:54:47 smithi032 ceph-mon[29763]: pgmap v582: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:54:49.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:54:49 smithi045 ceph-mon[33299]: pgmap v583: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:54:49.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:54:49 smithi113 ceph-mon[33540]: pgmap v583: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:54:49.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:54:49 smithi032 ceph-mon[29763]: pgmap v583: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:54:51.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:54:51 smithi045 ceph-mon[33299]: pgmap v584: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:54:51.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:54:51 smithi113 ceph-mon[33540]: pgmap v584: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:54:51.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:54:51 smithi032 ceph-mon[29763]: pgmap v584: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:54:53.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:54:53 smithi045 ceph-mon[33299]: pgmap v585: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:54:53.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:54:53 smithi113 ceph-mon[33540]: pgmap v585: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:54:53.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:54:53 smithi032 ceph-mon[29763]: pgmap v585: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:54:55.603 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:54:55 smithi113 ceph-mon[33540]: pgmap v586: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:54:55.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:54:55 smithi045 ceph-mon[33299]: pgmap v586: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:54:55.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:54:55 smithi032 ceph-mon[29763]: pgmap v586: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:54:57.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:54:57 smithi045 ceph-mon[33299]: pgmap v587: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:54:57.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:54:57 smithi113 ceph-mon[33540]: pgmap v587: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:54:57.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:54:57 smithi032 ceph-mon[29763]: pgmap v587: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:54:59.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:54:59 smithi045 ceph-mon[33299]: pgmap v588: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:54:59.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:54:59 smithi113 ceph-mon[33540]: pgmap v588: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:54:59.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:54:59 smithi032 ceph-mon[29763]: pgmap v588: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:55:01.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:55:01 smithi045 ceph-mon[33299]: pgmap v589: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:55:01.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:55:01 smithi113 ceph-mon[33540]: pgmap v589: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:55:01.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:55:01 smithi032 ceph-mon[29763]: pgmap v589: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:55:03.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:55:03 smithi045 ceph-mon[33299]: pgmap v590: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:55:03.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:55:03 smithi113 ceph-mon[33540]: pgmap v590: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:55:03.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:55:03 smithi032 ceph-mon[29763]: pgmap v590: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:55:05.248 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:55:04 smithi045 ceph-mon[33299]: pgmap v591: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:55:05.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:55:04 smithi113 ceph-mon[33540]: pgmap v591: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:55:05.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:55:04 smithi032 ceph-mon[29763]: pgmap v591: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:55:07.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:55:07 smithi045 ceph-mon[33299]: pgmap v592: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:55:07.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:55:07 smithi113 ceph-mon[33540]: pgmap v592: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:55:07.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:55:07 smithi032 ceph-mon[29763]: pgmap v592: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:55:09.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:55:09 smithi045 ceph-mon[33299]: pgmap v593: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:55:09.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:55:09 smithi113 ceph-mon[33540]: pgmap v593: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:55:09.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:55:09 smithi032 ceph-mon[29763]: pgmap v593: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:55:11.497 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:55:11 smithi045 ceph-mon[33299]: pgmap v594: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:55:11.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:55:11 smithi113 ceph-mon[33540]: pgmap v594: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:55:11.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:55:11 smithi032 ceph-mon[29763]: pgmap v594: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:55:12.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:55:12 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:55:12.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:55:12 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:55:12.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:55:12 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:55:12.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:55:12 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:55:12.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:55:12 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:55:12.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:55:12 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:55:12.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:55:12 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:55:12.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:55:12 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:55:12.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:55:12 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:55:12.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:55:12 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:55:12.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:55:12 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:55:12.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:55:12 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:55:13.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:55:13 smithi045 ceph-mon[33299]: pgmap v595: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:55:13.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:55:13 smithi113 ceph-mon[33540]: pgmap v595: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:55:13.569 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:55:13 smithi032 ceph-mon[29763]: pgmap v595: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:55:15.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:55:15 smithi045 ceph-mon[33299]: pgmap v596: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:55:15.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:55:15 smithi113 ceph-mon[33540]: pgmap v596: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:55:15.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:55:15 smithi032 ceph-mon[29763]: pgmap v596: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:55:17.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:55:17 smithi045 ceph-mon[33299]: pgmap v597: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:55:17.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:55:17 smithi113 ceph-mon[33540]: pgmap v597: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:55:17.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:55:17 smithi032 ceph-mon[29763]: pgmap v597: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:55:19.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:55:19 smithi045 ceph-mon[33299]: pgmap v598: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:55:19.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:55:19 smithi113 ceph-mon[33540]: pgmap v598: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:55:19.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:55:19 smithi032 ceph-mon[29763]: pgmap v598: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:55:21.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:55:21 smithi045 ceph-mon[33299]: pgmap v599: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:55:21.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:55:21 smithi113 ceph-mon[33540]: pgmap v599: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:55:21.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:55:21 smithi032 ceph-mon[29763]: pgmap v599: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:55:23.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:55:23 smithi045 ceph-mon[33299]: pgmap v600: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:55:23.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:55:23 smithi113 ceph-mon[33540]: pgmap v600: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:55:23.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:55:23 smithi032 ceph-mon[29763]: pgmap v600: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:55:25.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:55:25 smithi045 ceph-mon[33299]: pgmap v601: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:55:25.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:55:25 smithi113 ceph-mon[33540]: pgmap v601: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:55:25.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:55:25 smithi032 ceph-mon[29763]: pgmap v601: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:55:27.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:55:27 smithi045 ceph-mon[33299]: pgmap v602: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:55:27.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:55:27 smithi113 ceph-mon[33540]: pgmap v602: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:55:27.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:55:27 smithi032 ceph-mon[29763]: pgmap v602: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:55:29.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:55:29 smithi045 ceph-mon[33299]: pgmap v603: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:55:29.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:55:29 smithi113 ceph-mon[33540]: pgmap v603: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:55:29.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:55:29 smithi032 ceph-mon[29763]: pgmap v603: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:55:31.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:55:31 smithi045 ceph-mon[33299]: pgmap v604: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:55:31.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:55:31 smithi113 ceph-mon[33540]: pgmap v604: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:55:31.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:55:31 smithi032 ceph-mon[29763]: pgmap v604: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:55:33.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:55:33 smithi045 ceph-mon[33299]: pgmap v605: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:55:33.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:55:33 smithi113 ceph-mon[33540]: pgmap v605: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:55:33.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:55:33 smithi032 ceph-mon[29763]: pgmap v605: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:55:35.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:55:35 smithi045 ceph-mon[33299]: pgmap v606: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:55:35.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:55:35 smithi113 ceph-mon[33540]: pgmap v606: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:55:35.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:55:35 smithi032 ceph-mon[29763]: pgmap v606: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:55:37.498 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:55:37 smithi045 ceph-mon[33299]: pgmap v607: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:55:37.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:55:37 smithi113 ceph-mon[33540]: pgmap v607: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:55:37.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:55:37 smithi032 ceph-mon[29763]: pgmap v607: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:55:39.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:55:39 smithi113 ceph-mon[33540]: pgmap v608: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:55:39.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:55:39 smithi032 ceph-mon[29763]: pgmap v608: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:55:39.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:55:39 smithi045 ceph-mon[33299]: pgmap v608: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:55:41.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:55:41 smithi113 ceph-mon[33540]: pgmap v609: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:55:41.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:55:41 smithi032 ceph-mon[29763]: pgmap v609: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:55:41.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:55:41 smithi045 ceph-mon[33299]: pgmap v609: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:55:43.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:55:43 smithi113 ceph-mon[33540]: pgmap v610: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:55:43.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:55:43 smithi032 ceph-mon[29763]: pgmap v610: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:55:43.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:55:43 smithi045 ceph-mon[33299]: pgmap v610: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:55:45.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:55:45 smithi113 ceph-mon[33540]: pgmap v611: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:55:45.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:55:45 smithi032 ceph-mon[29763]: pgmap v611: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:55:45.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:55:45 smithi045 ceph-mon[33299]: pgmap v611: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:55:47.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:55:47 smithi113 ceph-mon[33540]: pgmap v612: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:55:47.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:55:47 smithi032 ceph-mon[29763]: pgmap v612: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:55:47.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:55:47 smithi045 ceph-mon[33299]: pgmap v612: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:55:49.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:55:49 smithi113 ceph-mon[33540]: pgmap v613: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:55:49.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:55:49 smithi032 ceph-mon[29763]: pgmap v613: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:55:49.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:55:49 smithi045 ceph-mon[33299]: pgmap v613: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:55:51.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:55:51 smithi113 ceph-mon[33540]: pgmap v614: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:55:51.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:55:51 smithi032 ceph-mon[29763]: pgmap v614: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:55:51.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:55:51 smithi045 ceph-mon[33299]: pgmap v614: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:55:53.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:55:53 smithi113 ceph-mon[33540]: pgmap v615: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:55:53.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:55:53 smithi032 ceph-mon[29763]: pgmap v615: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:55:53.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:55:53 smithi045 ceph-mon[33299]: pgmap v615: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:55:55.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:55:55 smithi113 ceph-mon[33540]: pgmap v616: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:55:55.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:55:55 smithi032 ceph-mon[29763]: pgmap v616: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:55:55.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:55:55 smithi045 ceph-mon[33299]: pgmap v616: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:55:57.550 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:55:57 smithi045 ceph-mon[33299]: pgmap v617: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:55:57.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:55:57 smithi113 ceph-mon[33540]: pgmap v617: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:55:57.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:55:57 smithi032 ceph-mon[29763]: pgmap v617: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:55:59.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:55:59 smithi113 ceph-mon[33540]: pgmap v618: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:55:59.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:55:59 smithi032 ceph-mon[29763]: pgmap v618: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:55:59.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:55:59 smithi045 ceph-mon[33299]: pgmap v618: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:56:01.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:56:01 smithi113 ceph-mon[33540]: pgmap v619: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:56:01.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:56:01 smithi032 ceph-mon[29763]: pgmap v619: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:56:01.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:56:01 smithi045 ceph-mon[33299]: pgmap v619: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:56:02.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:56:02.531+0000 7f9a0e7fc640 1 -- 172.21.15.32:0/3388640237 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- auth(proto 2 132 bytes epoch 0) -- 0x7f9a04005b70 con 0x7f9a20102b00 2024-05-03T18:56:02.532 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:56:02.531+0000 7f9a0f7fe640 1 -- 172.21.15.32:0/3388640237 <== mon.0 v2:172.21.15.32:3300/0 9 ==== auth_reply(proto 2 0 (0) Success) ==== 233+0+0 (secure 0 0 0) 0x7f9a04005b70 con 0x7f9a20102b00 2024-05-03T18:56:03.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:56:03 smithi113 ceph-mon[33540]: pgmap v620: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:56:03.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:56:03 smithi032 ceph-mon[29763]: pgmap v620: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:56:03.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:56:03 smithi045 ceph-mon[33299]: pgmap v620: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:56:05.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:56:05 smithi113 ceph-mon[33540]: pgmap v621: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:56:05.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:56:05 smithi032 ceph-mon[29763]: pgmap v621: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:56:05.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:56:05 smithi045 ceph-mon[33299]: pgmap v621: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:56:07.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:56:07 smithi113 ceph-mon[33540]: pgmap v622: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:56:07.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:56:07 smithi032 ceph-mon[29763]: pgmap v622: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:56:07.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:56:07 smithi045 ceph-mon[33299]: pgmap v622: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:56:09.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:56:09 smithi113 ceph-mon[33540]: pgmap v623: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:56:09.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:56:09 smithi032 ceph-mon[29763]: pgmap v623: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:56:09.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:56:09 smithi045 ceph-mon[33299]: pgmap v623: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:56:11.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:56:11 smithi113 ceph-mon[33540]: pgmap v624: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:56:11.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:56:11 smithi032 ceph-mon[29763]: pgmap v624: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:56:11.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:56:11 smithi045 ceph-mon[33299]: pgmap v624: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:56:12.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:56:12 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:56:12.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:56:12 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:56:12.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:56:12 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:56:13.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:56:13 smithi032 ceph-mon[29763]: pgmap v625: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:56:13.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:56:13 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:56:13.569 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:56:13 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:56:13.569 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:56:13 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:56:13.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:56:13 smithi045 ceph-mon[33299]: pgmap v625: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:56:13.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:56:13 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:56:13.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:56:13 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:56:13.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:56:13 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:56:13.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:56:13 smithi113 ceph-mon[33540]: pgmap v625: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:56:13.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:56:13 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:56:13.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:56:13 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:56:13.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:56:13 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:56:15.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:56:15 smithi032 ceph-mon[29763]: pgmap v626: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:56:15.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:56:15 smithi045 ceph-mon[33299]: pgmap v626: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:56:15.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:56:15 smithi113 ceph-mon[33540]: pgmap v626: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:56:17.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:56:17 smithi032 ceph-mon[29763]: pgmap v627: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:56:17.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:56:17 smithi045 ceph-mon[33299]: pgmap v627: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:56:17.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:56:17 smithi113 ceph-mon[33540]: pgmap v627: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:56:19.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:56:19 smithi032 ceph-mon[29763]: pgmap v628: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:56:19.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:56:19 smithi045 ceph-mon[33299]: pgmap v628: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:56:19.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:56:19 smithi113 ceph-mon[33540]: pgmap v628: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:56:21.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:56:21 smithi032 ceph-mon[29763]: pgmap v629: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:56:21.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:56:21 smithi045 ceph-mon[33299]: pgmap v629: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:56:21.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:56:21 smithi113 ceph-mon[33540]: pgmap v629: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:56:23.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:56:23 smithi032 ceph-mon[29763]: pgmap v630: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:56:23.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:56:23 smithi045 ceph-mon[33299]: pgmap v630: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:56:23.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:56:23 smithi113 ceph-mon[33540]: pgmap v630: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:56:25.603 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:56:25 smithi113 ceph-mon[33540]: pgmap v631: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:56:25.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:56:25 smithi045 ceph-mon[33299]: pgmap v631: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:56:25.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:56:25 smithi032 ceph-mon[29763]: pgmap v631: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:56:27.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:56:27 smithi045 ceph-mon[33299]: pgmap v632: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:56:27.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:56:27 smithi113 ceph-mon[33540]: pgmap v632: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:56:27.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:56:27 smithi032 ceph-mon[29763]: pgmap v632: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:56:29.750 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:56:29 smithi045 ceph-mon[33299]: pgmap v633: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:56:29.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:56:29 smithi113 ceph-mon[33540]: pgmap v633: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:56:29.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:56:29 smithi032 ceph-mon[29763]: pgmap v633: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:56:31.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:56:31 smithi045 ceph-mon[33299]: pgmap v634: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:56:31.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:56:31 smithi113 ceph-mon[33540]: pgmap v634: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:56:31.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:56:31 smithi032 ceph-mon[29763]: pgmap v634: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:56:33.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:56:33 smithi045 ceph-mon[33299]: pgmap v635: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-05-03T18:56:33.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:56:33 smithi113 ceph-mon[33540]: pgmap v635: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-05-03T18:56:33.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:56:33 smithi032 ceph-mon[29763]: pgmap v635: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-05-03T18:56:35.620 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:56:35 smithi032 ceph-mon[29763]: pgmap v636: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-05-03T18:56:35.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:56:35 smithi045 ceph-mon[33299]: pgmap v636: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-05-03T18:56:35.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:56:35 smithi113 ceph-mon[33540]: pgmap v636: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-05-03T18:56:37.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:56:37 smithi045 ceph-mon[33299]: pgmap v637: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-05-03T18:56:37.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:56:37 smithi113 ceph-mon[33540]: pgmap v637: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-05-03T18:56:37.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:56:37 smithi032 ceph-mon[29763]: pgmap v637: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-05-03T18:56:39.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:56:39 smithi045 ceph-mon[33299]: pgmap v638: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-05-03T18:56:39.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:56:39 smithi113 ceph-mon[33540]: pgmap v638: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-05-03T18:56:39.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:56:39 smithi032 ceph-mon[29763]: pgmap v638: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-05-03T18:56:41.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:56:41 smithi045 ceph-mon[33299]: pgmap v639: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-05-03T18:56:41.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:56:41 smithi113 ceph-mon[33540]: pgmap v639: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-05-03T18:56:41.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:56:41 smithi032 ceph-mon[29763]: pgmap v639: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-05-03T18:56:43.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:56:43 smithi045 ceph-mon[33299]: pgmap v640: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-05-03T18:56:43.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:56:43 smithi113 ceph-mon[33540]: pgmap v640: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-05-03T18:56:43.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:56:43 smithi032 ceph-mon[29763]: pgmap v640: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-05-03T18:56:45.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:56:45 smithi045 ceph-mon[33299]: pgmap v641: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:56:45.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:56:45 smithi113 ceph-mon[33540]: pgmap v641: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:56:45.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:56:45 smithi032 ceph-mon[29763]: pgmap v641: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:56:47.538 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:56:47.537+0000 7f9a0e7fc640 1 -- 172.21.15.32:0/3388640237 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7f9a04005ec0 con 0x7f9a20102b00 2024-05-03T18:56:47.539 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:56:47.537+0000 7f9a0f7fe640 1 -- 172.21.15.32:0/3388640237 <== mon.0 v2:172.21.15.32:3300/0 10 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7f9a04005ec0 con 0x7f9a20102b00 2024-05-03T18:56:47.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:56:47 smithi045 ceph-mon[33299]: pgmap v642: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:56:47.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:56:47 smithi113 ceph-mon[33540]: pgmap v642: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:56:47.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:56:47 smithi032 ceph-mon[29763]: pgmap v642: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:56:49.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:56:49 smithi045 ceph-mon[33299]: pgmap v643: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:56:49.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:56:49 smithi113 ceph-mon[33540]: pgmap v643: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:56:49.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:56:49 smithi032 ceph-mon[29763]: pgmap v643: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:56:51.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:56:51 smithi045 ceph-mon[33299]: pgmap v644: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:56:51.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:56:51 smithi113 ceph-mon[33540]: pgmap v644: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:56:51.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:56:51 smithi032 ceph-mon[29763]: pgmap v644: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:56:53.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:56:53 smithi045 ceph-mon[33299]: pgmap v645: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:56:53.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:56:53 smithi113 ceph-mon[33540]: pgmap v645: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:56:53.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:56:53 smithi032 ceph-mon[29763]: pgmap v645: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:56:55.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:56:55 smithi045 ceph-mon[33299]: pgmap v646: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:56:55.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:56:55 smithi113 ceph-mon[33540]: pgmap v646: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:56:55.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:56:55 smithi032 ceph-mon[29763]: pgmap v646: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:56:57.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:56:57 smithi045 ceph-mon[33299]: pgmap v647: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:56:57.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:56:57 smithi113 ceph-mon[33540]: pgmap v647: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:56:57.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:56:57 smithi032 ceph-mon[29763]: pgmap v647: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:56:59.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:56:59 smithi045 ceph-mon[33299]: pgmap v648: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:56:59.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:56:59 smithi113 ceph-mon[33540]: pgmap v648: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:56:59.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:56:59 smithi032 ceph-mon[29763]: pgmap v648: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:57:01.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:57:01 smithi045 ceph-mon[33299]: pgmap v649: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:57:01.805 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:57:01 smithi113 ceph-mon[33540]: pgmap v649: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:57:01.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:57:01 smithi032 ceph-mon[29763]: pgmap v649: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:57:03.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:57:03 smithi045 ceph-mon[33299]: pgmap v650: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:57:03.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:57:03 smithi113 ceph-mon[33540]: pgmap v650: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:57:03.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:57:03 smithi032 ceph-mon[29763]: pgmap v650: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:57:05.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:57:05 smithi045 ceph-mon[33299]: pgmap v651: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:57:05.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:57:05 smithi113 ceph-mon[33540]: pgmap v651: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:57:05.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:57:05 smithi032 ceph-mon[29763]: pgmap v651: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:57:07.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:57:07 smithi045 ceph-mon[33299]: pgmap v652: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:57:07.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:57:07 smithi113 ceph-mon[33540]: pgmap v652: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:57:07.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:57:07 smithi032 ceph-mon[29763]: pgmap v652: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:57:09.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:57:09 smithi045 ceph-mon[33299]: pgmap v653: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-05-03T18:57:09.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:57:09 smithi113 ceph-mon[33540]: pgmap v653: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-05-03T18:57:09.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:57:09 smithi032 ceph-mon[29763]: pgmap v653: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-05-03T18:57:11.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:57:11 smithi045 ceph-mon[33299]: pgmap v654: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-05-03T18:57:11.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:57:11 smithi113 ceph-mon[33540]: pgmap v654: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-05-03T18:57:11.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:57:11 smithi032 ceph-mon[29763]: pgmap v654: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-05-03T18:57:13.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:57:13 smithi045 ceph-mon[33299]: pgmap v655: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-05-03T18:57:13.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:57:13 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:57:13.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:57:13 smithi113 ceph-mon[33540]: pgmap v655: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-05-03T18:57:13.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:57:13 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:57:13.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:57:13 smithi032 ceph-mon[29763]: pgmap v655: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-05-03T18:57:13.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:57:13 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:57:15.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:57:15 smithi045 ceph-mon[33299]: pgmap v656: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-05-03T18:57:15.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:57:15 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:57:15.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:57:15 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:57:15.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:57:15 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:57:15.999 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:57:15 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:57:15.999 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:57:15 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:57:15.999 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:57:15 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:57:15.999 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:57:15 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:57:15.999 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:57:15 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:57:15.999 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:57:15 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:57:16.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:57:15 smithi113 ceph-mon[33540]: pgmap v656: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-05-03T18:57:16.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:57:15 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:57:16.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:57:15 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:57:16.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:57:15 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:57:16.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:57:15 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:57:16.052 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:57:15 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:57:16.052 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:57:15 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:57:16.052 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:57:15 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:57:16.052 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:57:15 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:57:16.052 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:57:15 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:57:16.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:57:15 smithi032 ceph-mon[29763]: pgmap v656: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-05-03T18:57:16.069 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:57:15 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:57:16.069 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:57:15 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:57:16.069 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:57:15 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:57:16.069 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:57:15 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:57:16.069 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:57:15 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:57:16.069 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:57:15 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:57:16.069 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:57:15 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:57:16.069 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:57:15 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:57:16.069 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:57:15 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:57:17.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:57:17 smithi045 ceph-mon[33299]: pgmap v657: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-05-03T18:57:18.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:57:17 smithi113 ceph-mon[33540]: pgmap v657: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-05-03T18:57:18.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:57:17 smithi032 ceph-mon[29763]: pgmap v657: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-05-03T18:57:19.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:57:19 smithi045 ceph-mon[33299]: pgmap v658: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-05-03T18:57:20.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:57:19 smithi113 ceph-mon[33540]: pgmap v658: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-05-03T18:57:20.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:57:19 smithi032 ceph-mon[29763]: pgmap v658: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-05-03T18:57:21.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:57:21 smithi045 ceph-mon[33299]: pgmap v659: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:57:22.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:57:21 smithi113 ceph-mon[33540]: pgmap v659: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:57:22.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:57:21 smithi032 ceph-mon[29763]: pgmap v659: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:57:23.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:57:23 smithi045 ceph-mon[33299]: pgmap v660: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:57:24.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:57:23 smithi113 ceph-mon[33540]: pgmap v660: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:57:24.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:57:23 smithi032 ceph-mon[29763]: pgmap v660: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:57:25.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:57:25 smithi045 ceph-mon[33299]: pgmap v661: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:57:26.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:57:25 smithi113 ceph-mon[33540]: pgmap v661: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:57:26.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:57:25 smithi032 ceph-mon[29763]: pgmap v661: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:57:27.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:57:27 smithi045 ceph-mon[33299]: pgmap v662: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:57:28.052 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:57:27 smithi113 ceph-mon[33540]: pgmap v662: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:57:28.069 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:57:27 smithi032 ceph-mon[29763]: pgmap v662: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:57:29.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:57:29 smithi045 ceph-mon[33299]: pgmap v663: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:57:30.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:57:29 smithi113 ceph-mon[33540]: pgmap v663: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:57:30.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:57:29 smithi032 ceph-mon[29763]: pgmap v663: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:57:31.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:57:31 smithi045 ceph-mon[33299]: pgmap v664: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:57:32.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:57:31 smithi113 ceph-mon[33540]: pgmap v664: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:57:32.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:57:31 smithi032 ceph-mon[29763]: pgmap v664: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:57:33.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:57:33 smithi045 ceph-mon[33299]: pgmap v665: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:57:34.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:57:33 smithi113 ceph-mon[33540]: pgmap v665: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:57:34.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:57:33 smithi032 ceph-mon[29763]: pgmap v665: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:57:35.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:57:35 smithi045 ceph-mon[33299]: pgmap v666: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:57:36.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:57:35 smithi113 ceph-mon[33540]: pgmap v666: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:57:36.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:57:35 smithi032 ceph-mon[29763]: pgmap v666: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:57:37.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:57:37 smithi045 ceph-mon[33299]: pgmap v667: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:57:38.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:57:37 smithi113 ceph-mon[33540]: pgmap v667: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:57:38.069 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:57:37 smithi032 ceph-mon[29763]: pgmap v667: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:57:39.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:57:39 smithi045 ceph-mon[33299]: pgmap v668: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:57:40.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:57:39 smithi113 ceph-mon[33540]: pgmap v668: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:57:40.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:57:39 smithi032 ceph-mon[29763]: pgmap v668: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:57:41.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:57:41 smithi045 ceph-mon[33299]: pgmap v669: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:57:42.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:57:41 smithi113 ceph-mon[33540]: pgmap v669: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:57:42.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:57:41 smithi032 ceph-mon[29763]: pgmap v669: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:57:43.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:57:43 smithi045 ceph-mon[33299]: pgmap v670: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:57:44.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:57:43 smithi113 ceph-mon[33540]: pgmap v670: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:57:44.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:57:43 smithi032 ceph-mon[29763]: pgmap v670: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:57:45.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:57:45 smithi045 ceph-mon[33299]: pgmap v671: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:57:46.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:57:45 smithi113 ceph-mon[33540]: pgmap v671: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:57:46.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:57:45 smithi032 ceph-mon[29763]: pgmap v671: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:57:47.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:57:47 smithi045 ceph-mon[33299]: pgmap v672: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:57:48.052 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:57:47 smithi113 ceph-mon[33540]: pgmap v672: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:57:48.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:57:47 smithi032 ceph-mon[29763]: pgmap v672: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:57:49.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:57:49 smithi045 ceph-mon[33299]: pgmap v673: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:57:50.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:57:49 smithi113 ceph-mon[33540]: pgmap v673: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:57:50.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:57:49 smithi032 ceph-mon[29763]: pgmap v673: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:57:51.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:57:51 smithi045 ceph-mon[33299]: pgmap v674: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:57:52.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:57:51 smithi113 ceph-mon[33540]: pgmap v674: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:57:52.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:57:51 smithi032 ceph-mon[29763]: pgmap v674: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:57:53.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:57:53 smithi045 ceph-mon[33299]: pgmap v675: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:57:54.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:57:53 smithi113 ceph-mon[33540]: pgmap v675: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:57:54.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:57:53 smithi032 ceph-mon[29763]: pgmap v675: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:57:55.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:57:55 smithi045 ceph-mon[33299]: pgmap v676: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:57:56.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:57:55 smithi113 ceph-mon[33540]: pgmap v676: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:57:56.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:57:55 smithi032 ceph-mon[29763]: pgmap v676: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:57:57.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:57:57 smithi045 ceph-mon[33299]: pgmap v677: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:57:58.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:57:57 smithi113 ceph-mon[33540]: pgmap v677: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:57:58.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:57:57 smithi032 ceph-mon[29763]: pgmap v677: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:57:59.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:57:59 smithi045 ceph-mon[33299]: pgmap v678: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:58:00.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:57:59 smithi113 ceph-mon[33540]: pgmap v678: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:58:00.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:57:59 smithi032 ceph-mon[29763]: pgmap v678: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:58:01.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:58:01 smithi045 ceph-mon[33299]: pgmap v679: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:58:02.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:58:01 smithi113 ceph-mon[33540]: pgmap v679: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:58:02.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:58:01 smithi032 ceph-mon[29763]: pgmap v679: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:58:03.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:58:03 smithi045 ceph-mon[33299]: pgmap v680: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:58:04.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:58:03 smithi113 ceph-mon[33540]: pgmap v680: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:58:04.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:58:03 smithi032 ceph-mon[29763]: pgmap v680: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:58:05.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:58:05 smithi045 ceph-mon[33299]: pgmap v681: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:58:06.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:58:05 smithi113 ceph-mon[33540]: pgmap v681: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:58:06.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:58:05 smithi032 ceph-mon[29763]: pgmap v681: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:58:07.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:58:07 smithi045 ceph-mon[33299]: pgmap v682: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:58:08.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:58:07 smithi113 ceph-mon[33540]: pgmap v682: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:58:08.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:58:07 smithi032 ceph-mon[29763]: pgmap v682: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:58:09.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:58:09 smithi045 ceph-mon[33299]: pgmap v683: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:58:10.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:58:09 smithi113 ceph-mon[33540]: pgmap v683: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:58:10.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:58:09 smithi032 ceph-mon[29763]: pgmap v683: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:58:11.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:58:11 smithi045 ceph-mon[33299]: pgmap v684: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:58:12.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:58:11 smithi113 ceph-mon[33540]: pgmap v684: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:58:12.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:58:11 smithi032 ceph-mon[29763]: pgmap v684: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:58:13.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:58:13 smithi045 ceph-mon[33299]: pgmap v685: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:58:14.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:58:13 smithi113 ceph-mon[33540]: pgmap v685: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:58:14.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:58:13 smithi032 ceph-mon[29763]: pgmap v685: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:58:15.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:58:15 smithi045 ceph-mon[33299]: pgmap v686: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:58:15.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:58:15 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:58:16.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:58:15 smithi113 ceph-mon[33540]: pgmap v686: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:58:16.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:58:15 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:58:16.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:58:15 smithi032 ceph-mon[29763]: pgmap v686: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:58:16.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:58:15 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:58:17.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:58:17 smithi113 ceph-mon[33540]: pgmap v687: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:58:17.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:58:17 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:58:17.552 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:58:17 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:58:17.552 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:58:17 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:58:17.552 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:58:17 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:58:17.552 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:58:17 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:58:17.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:58:17 smithi032 ceph-mon[29763]: pgmap v687: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:58:17.569 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:58:17 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:58:17.569 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:58:17 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:58:17.569 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:58:17 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:58:17.569 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:58:17 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:58:17.569 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:58:17 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:58:17.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:58:17 smithi045 ceph-mon[33299]: pgmap v687: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:58:17.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:58:17 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:58:17.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:58:17 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:58:17.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:58:17 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:58:17.749 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:58:17 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:58:17.749 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:58:17 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:58:19.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:58:19 smithi113 ceph-mon[33540]: pgmap v688: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:58:19.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:58:19 smithi032 ceph-mon[29763]: pgmap v688: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:58:19.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:58:19 smithi045 ceph-mon[33299]: pgmap v688: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:58:21.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:58:21 smithi113 ceph-mon[33540]: pgmap v689: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:58:21.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:58:21 smithi032 ceph-mon[29763]: pgmap v689: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:58:21.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:58:21 smithi045 ceph-mon[33299]: pgmap v689: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:58:23.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:58:23 smithi113 ceph-mon[33540]: pgmap v690: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:58:23.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:58:23 smithi032 ceph-mon[29763]: pgmap v690: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:58:23.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:58:23 smithi045 ceph-mon[33299]: pgmap v690: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:58:25.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:58:25 smithi113 ceph-mon[33540]: pgmap v691: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:58:25.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:58:25 smithi032 ceph-mon[29763]: pgmap v691: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:58:25.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:58:25 smithi045 ceph-mon[33299]: pgmap v691: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:58:27.550 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:58:27 smithi045 ceph-mon[33299]: pgmap v692: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:58:27.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:58:27 smithi113 ceph-mon[33540]: pgmap v692: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:58:27.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:58:27 smithi032 ceph-mon[29763]: pgmap v692: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:58:29.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:58:29 smithi113 ceph-mon[33540]: pgmap v693: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:58:29.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:58:29 smithi032 ceph-mon[29763]: pgmap v693: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:58:29.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:58:29 smithi045 ceph-mon[33299]: pgmap v693: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:58:31.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:58:31 smithi113 ceph-mon[33540]: pgmap v694: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:58:31.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:58:31 smithi032 ceph-mon[29763]: pgmap v694: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:58:31.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:58:31 smithi045 ceph-mon[33299]: pgmap v694: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:58:33.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:58:33 smithi113 ceph-mon[33540]: pgmap v695: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:58:33.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:58:33 smithi032 ceph-mon[29763]: pgmap v695: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:58:33.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:58:33 smithi045 ceph-mon[33299]: pgmap v695: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:58:35.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:58:35 smithi113 ceph-mon[33540]: pgmap v696: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:58:35.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:58:35 smithi032 ceph-mon[29763]: pgmap v696: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:58:35.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:58:35 smithi045 ceph-mon[33299]: pgmap v696: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:58:37.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:58:37 smithi113 ceph-mon[33540]: pgmap v697: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:58:37.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:58:37 smithi032 ceph-mon[29763]: pgmap v697: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:58:37.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:58:37 smithi045 ceph-mon[33299]: pgmap v697: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:58:39.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:58:39 smithi113 ceph-mon[33540]: pgmap v698: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:58:39.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:58:39 smithi032 ceph-mon[29763]: pgmap v698: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:58:39.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:58:39 smithi045 ceph-mon[33299]: pgmap v698: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:58:41.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:58:41 smithi113 ceph-mon[33540]: pgmap v699: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:58:41.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:58:41 smithi032 ceph-mon[29763]: pgmap v699: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:58:41.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:58:41 smithi045 ceph-mon[33299]: pgmap v699: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:58:43.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:58:43 smithi113 ceph-mon[33540]: pgmap v700: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:58:43.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:58:43 smithi032 ceph-mon[29763]: pgmap v700: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:58:43.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:58:43 smithi045 ceph-mon[33299]: pgmap v700: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:58:45.248 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:58:44 smithi045 ceph-mon[33299]: pgmap v701: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:58:45.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:58:44 smithi113 ceph-mon[33540]: pgmap v701: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:58:45.318 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:58:44 smithi032 ceph-mon[29763]: pgmap v701: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:58:47.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:58:47 smithi113 ceph-mon[33540]: pgmap v702: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:58:47.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:58:47 smithi032 ceph-mon[29763]: pgmap v702: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:58:47.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:58:47 smithi045 ceph-mon[33299]: pgmap v702: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:58:49.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:58:49 smithi113 ceph-mon[33540]: pgmap v703: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:58:49.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:58:49 smithi032 ceph-mon[29763]: pgmap v703: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:58:49.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:58:49 smithi045 ceph-mon[33299]: pgmap v703: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:58:51.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:58:51 smithi113 ceph-mon[33540]: pgmap v704: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:58:51.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:58:51 smithi032 ceph-mon[29763]: pgmap v704: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:58:51.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:58:51 smithi045 ceph-mon[33299]: pgmap v704: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:58:53.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:58:53 smithi113 ceph-mon[33540]: pgmap v705: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:58:53.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:58:53 smithi032 ceph-mon[29763]: pgmap v705: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:58:53.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:58:53 smithi045 ceph-mon[33299]: pgmap v705: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:58:55.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:58:55 smithi113 ceph-mon[33540]: pgmap v706: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:58:55.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:58:55 smithi032 ceph-mon[29763]: pgmap v706: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:58:55.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:58:55 smithi045 ceph-mon[33299]: pgmap v706: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:58:57.550 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:58:57 smithi045 ceph-mon[33299]: pgmap v707: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:58:57.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:58:57 smithi113 ceph-mon[33540]: pgmap v707: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:58:57.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:58:57 smithi032 ceph-mon[29763]: pgmap v707: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:58:59.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:58:59 smithi113 ceph-mon[33540]: pgmap v708: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:58:59.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:58:59 smithi032 ceph-mon[29763]: pgmap v708: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:58:59.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:58:59 smithi045 ceph-mon[33299]: pgmap v708: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:59:01.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:59:01 smithi113 ceph-mon[33540]: pgmap v709: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:59:01.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:59:01 smithi032 ceph-mon[29763]: pgmap v709: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:59:01.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:59:01 smithi045 ceph-mon[33299]: pgmap v709: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:59:03.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:59:03 smithi113 ceph-mon[33540]: pgmap v710: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:59:03.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:59:03 smithi032 ceph-mon[29763]: pgmap v710: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:59:03.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:59:03 smithi045 ceph-mon[33299]: pgmap v710: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:59:05.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:59:05 smithi113 ceph-mon[33540]: pgmap v711: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:59:05.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:59:05 smithi032 ceph-mon[29763]: pgmap v711: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:59:05.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:59:05 smithi045 ceph-mon[33299]: pgmap v711: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:59:07.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:59:07 smithi113 ceph-mon[33540]: pgmap v712: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:59:07.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:59:07 smithi032 ceph-mon[29763]: pgmap v712: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:59:07.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:59:07 smithi045 ceph-mon[33299]: pgmap v712: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:59:09.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:59:09 smithi113 ceph-mon[33540]: pgmap v713: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:59:09.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:59:09 smithi032 ceph-mon[29763]: pgmap v713: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:59:09.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:59:09 smithi045 ceph-mon[33299]: pgmap v713: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:59:11.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:59:11 smithi032 ceph-mon[29763]: pgmap v714: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:59:11.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:59:11 smithi045 ceph-mon[33299]: pgmap v714: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:59:11.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:59:11 smithi113 ceph-mon[33540]: pgmap v714: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:59:13.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:59:13 smithi032 ceph-mon[29763]: pgmap v715: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:59:13.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:59:13 smithi045 ceph-mon[33299]: pgmap v715: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:59:13.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:59:13 smithi113 ceph-mon[33540]: pgmap v715: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:59:15.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:59:15 smithi032 ceph-mon[29763]: pgmap v716: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:59:15.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:59:15 smithi045 ceph-mon[33299]: pgmap v716: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:59:15.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:59:15 smithi113 ceph-mon[33540]: pgmap v716: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:59:17.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:59:17 smithi032 ceph-mon[29763]: pgmap v717: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:59:17.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:59:17 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:59:17.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:59:17 smithi045 ceph-mon[33299]: pgmap v717: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:59:17.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:59:17 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:59:17.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:59:17 smithi113 ceph-mon[33540]: pgmap v717: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:59:17.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:59:17 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T18:59:18.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:59:18 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:59:18.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:59:18 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:59:18.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:59:18 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:59:18.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:59:18 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:59:18.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:59:18 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:59:18.999 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:59:18 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:59:18.999 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:59:18 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:59:19.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:59:18 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:59:19.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:59:18 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:59:19.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:59:18 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:59:19.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:59:18 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:59:19.052 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:59:18 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:59:19.052 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:59:18 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:59:19.052 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:59:18 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:59:19.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:59:18 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:59:19.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:59:18 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:59:19.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:59:18 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:59:19.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:59:18 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:59:19.069 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:59:18 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T18:59:19.069 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:59:18 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T18:59:19.069 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:59:18 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T18:59:19.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:59:19 smithi045 ceph-mon[33299]: pgmap v718: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:59:20.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:59:19 smithi113 ceph-mon[33540]: pgmap v718: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:59:20.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:59:19 smithi032 ceph-mon[29763]: pgmap v718: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:59:21.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:59:21 smithi045 ceph-mon[33299]: pgmap v719: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:59:22.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:59:21 smithi113 ceph-mon[33540]: pgmap v719: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:59:22.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:59:21 smithi032 ceph-mon[29763]: pgmap v719: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:59:23.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:59:23 smithi045 ceph-mon[33299]: pgmap v720: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:59:24.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:59:23 smithi113 ceph-mon[33540]: pgmap v720: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:59:24.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:59:23 smithi032 ceph-mon[29763]: pgmap v720: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:59:25.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:59:25 smithi045 ceph-mon[33299]: pgmap v721: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:59:26.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:59:25 smithi113 ceph-mon[33540]: pgmap v721: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:59:26.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:59:25 smithi032 ceph-mon[29763]: pgmap v721: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:59:27.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:59:27 smithi045 ceph-mon[33299]: pgmap v722: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:59:28.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:59:27 smithi113 ceph-mon[33540]: pgmap v722: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:59:28.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:59:27 smithi032 ceph-mon[29763]: pgmap v722: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:59:29.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:59:29 smithi045 ceph-mon[33299]: pgmap v723: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:59:30.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:59:29 smithi113 ceph-mon[33540]: pgmap v723: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:59:30.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:59:29 smithi032 ceph-mon[29763]: pgmap v723: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:59:31.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:59:31 smithi045 ceph-mon[33299]: pgmap v724: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:59:32.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:59:31 smithi113 ceph-mon[33540]: pgmap v724: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:59:32.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:59:31 smithi032 ceph-mon[29763]: pgmap v724: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:59:33.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:59:33 smithi045 ceph-mon[33299]: pgmap v725: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:59:34.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:59:33 smithi113 ceph-mon[33540]: pgmap v725: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:59:34.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:59:33 smithi032 ceph-mon[29763]: pgmap v725: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:59:35.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:59:35 smithi045 ceph-mon[33299]: pgmap v726: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:59:36.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:59:35 smithi113 ceph-mon[33540]: pgmap v726: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:59:36.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:59:35 smithi032 ceph-mon[29763]: pgmap v726: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:59:37.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:59:37 smithi045 ceph-mon[33299]: pgmap v727: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:59:38.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:59:37 smithi113 ceph-mon[33540]: pgmap v727: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:59:38.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:59:37 smithi032 ceph-mon[29763]: pgmap v727: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:59:39.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:59:39 smithi045 ceph-mon[33299]: pgmap v728: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:59:40.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:59:39 smithi113 ceph-mon[33540]: pgmap v728: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:59:40.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:59:39 smithi032 ceph-mon[29763]: pgmap v728: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:59:41.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:59:41 smithi045 ceph-mon[33299]: pgmap v729: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:59:42.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:59:41 smithi113 ceph-mon[33540]: pgmap v729: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:59:42.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:59:41 smithi032 ceph-mon[29763]: pgmap v729: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:59:43.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:59:43 smithi045 ceph-mon[33299]: pgmap v730: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:59:44.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:59:43 smithi113 ceph-mon[33540]: pgmap v730: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:59:44.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:59:43 smithi032 ceph-mon[29763]: pgmap v730: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:59:45.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:59:45 smithi045 ceph-mon[33299]: pgmap v731: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:59:46.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:59:45 smithi113 ceph-mon[33540]: pgmap v731: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:59:46.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:59:45 smithi032 ceph-mon[29763]: pgmap v731: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:59:47.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:59:47.563+0000 7f9a0e7fc640 1 -- 172.21.15.32:0/3388640237 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7f9a04006090 con 0x7f9a20102b00 2024-05-03T18:59:47.564 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T18:59:47.563+0000 7f9a0f7fe640 1 -- 172.21.15.32:0/3388640237 <== mon.0 v2:172.21.15.32:3300/0 11 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7f9a04006090 con 0x7f9a20102b00 2024-05-03T18:59:47.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:59:47 smithi045 ceph-mon[33299]: pgmap v732: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:59:48.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:59:47 smithi113 ceph-mon[33540]: pgmap v732: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:59:48.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:59:47 smithi032 ceph-mon[29763]: pgmap v732: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:59:49.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:59:49 smithi045 ceph-mon[33299]: pgmap v733: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:59:50.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:59:49 smithi113 ceph-mon[33540]: pgmap v733: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:59:50.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:59:49 smithi032 ceph-mon[29763]: pgmap v733: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:59:51.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:59:51 smithi045 ceph-mon[33299]: pgmap v734: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:59:52.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:59:51 smithi113 ceph-mon[33540]: pgmap v734: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:59:52.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:59:51 smithi032 ceph-mon[29763]: pgmap v734: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T18:59:53.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:59:53 smithi045 ceph-mon[33299]: pgmap v735: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:59:54.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:59:53 smithi113 ceph-mon[33540]: pgmap v735: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:59:54.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:59:53 smithi032 ceph-mon[29763]: pgmap v735: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:59:55.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:59:55 smithi045 ceph-mon[33299]: pgmap v736: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:59:56.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:59:55 smithi113 ceph-mon[33540]: pgmap v736: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:59:56.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:59:55 smithi032 ceph-mon[29763]: pgmap v736: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T18:59:57.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:59:57 smithi045 ceph-mon[33299]: pgmap v737: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:59:58.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:59:57 smithi113 ceph-mon[33540]: pgmap v737: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:59:58.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:59:57 smithi032 ceph-mon[29763]: pgmap v737: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T18:59:59.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 18:59:59 smithi045 ceph-mon[33299]: pgmap v738: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T19:00:00.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 18:59:59 smithi113 ceph-mon[33540]: pgmap v738: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T19:00:00.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 18:59:59 smithi032 ceph-mon[29763]: pgmap v738: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T19:00:00.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 19:00:00 smithi045 ceph-mon[33299]: overall HEALTH_OK 2024-05-03T19:00:00.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 19:00:00 smithi045 ceph-mon[33299]: pgmap v739: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T19:00:01.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 19:00:00 smithi113 ceph-mon[33540]: overall HEALTH_OK 2024-05-03T19:00:01.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 19:00:00 smithi113 ceph-mon[33540]: pgmap v739: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T19:00:01.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:00:00 smithi032 ceph-mon[29763]: overall HEALTH_OK 2024-05-03T19:00:01.068 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:00:00 smithi032 ceph-mon[29763]: pgmap v739: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T19:00:03.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 19:00:03 smithi113 ceph-mon[33540]: pgmap v740: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T19:00:03.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:00:03 smithi032 ceph-mon[29763]: pgmap v740: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T19:00:03.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 19:00:03 smithi045 ceph-mon[33299]: pgmap v740: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T19:00:05.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 19:00:05 smithi113 ceph-mon[33540]: pgmap v741: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T19:00:05.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:00:05 smithi032 ceph-mon[29763]: pgmap v741: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T19:00:05.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 19:00:05 smithi045 ceph-mon[33299]: pgmap v741: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T19:00:07.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 19:00:07 smithi113 ceph-mon[33540]: pgmap v742: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T19:00:07.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:00:07 smithi032 ceph-mon[29763]: pgmap v742: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T19:00:07.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 19:00:07 smithi045 ceph-mon[33299]: pgmap v742: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T19:00:09.551 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 19:00:09 smithi113 ceph-mon[33540]: pgmap v743: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T19:00:09.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:00:09 smithi032 ceph-mon[29763]: pgmap v743: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T19:00:09.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 19:00:09 smithi045 ceph-mon[33299]: pgmap v743: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T19:00:11.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:00:11 smithi032 ceph-mon[29763]: pgmap v744: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T19:00:11.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 19:00:11 smithi045 ceph-mon[33299]: pgmap v744: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T19:00:11.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 19:00:11 smithi113 ceph-mon[33540]: pgmap v744: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T19:00:13.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:00:13 smithi032 ceph-mon[29763]: pgmap v745: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T19:00:13.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 19:00:13 smithi045 ceph-mon[33299]: pgmap v745: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T19:00:13.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 19:00:13 smithi113 ceph-mon[33540]: pgmap v745: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T19:00:15.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:00:15 smithi032 ceph-mon[29763]: pgmap v746: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T19:00:15.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 19:00:15 smithi045 ceph-mon[33299]: pgmap v746: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T19:00:15.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 19:00:15 smithi113 ceph-mon[33540]: pgmap v746: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T19:00:17.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:00:17 smithi032 ceph-mon[29763]: pgmap v747: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T19:00:17.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 19:00:17 smithi045 ceph-mon[33299]: pgmap v747: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T19:00:17.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 19:00:17 smithi113 ceph-mon[33540]: pgmap v747: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T19:00:19.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:00:19 smithi032 ceph-mon[29763]: pgmap v748: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T19:00:19.569 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:00:19 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T19:00:19.569 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:00:19 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T19:00:19.569 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:00:19 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T19:00:19.569 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:00:19 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T19:00:19.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 19:00:19 smithi045 ceph-mon[33299]: pgmap v748: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T19:00:19.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 19:00:19 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T19:00:19.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 19:00:19 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T19:00:19.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 19:00:19 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T19:00:19.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 19:00:19 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T19:00:19.802 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 19:00:19 smithi113 ceph-mon[33540]: pgmap v748: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T19:00:19.802 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 19:00:19 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T19:00:19.802 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 19:00:19 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T19:00:19.802 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 19:00:19 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T19:00:19.802 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 19:00:19 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T19:00:21.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:00:21 smithi032 ceph-mon[29763]: pgmap v749: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T19:00:21.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 19:00:21 smithi045 ceph-mon[33299]: pgmap v749: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T19:00:21.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 19:00:21 smithi113 ceph-mon[33540]: pgmap v749: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T19:00:23.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 19:00:23 smithi045 ceph-mon[33299]: pgmap v750: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T19:00:23.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 19:00:23 smithi113 ceph-mon[33540]: pgmap v750: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T19:00:23.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:00:23 smithi032 ceph-mon[29763]: pgmap v750: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T19:00:25.603 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 19:00:25 smithi113 ceph-mon[33540]: pgmap v751: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T19:00:25.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 19:00:25 smithi045 ceph-mon[33299]: pgmap v751: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T19:00:25.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:00:25 smithi032 ceph-mon[29763]: pgmap v751: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T19:00:27.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 19:00:27 smithi045 ceph-mon[33299]: pgmap v752: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T19:00:27.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 19:00:27 smithi113 ceph-mon[33540]: pgmap v752: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T19:00:27.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:00:27 smithi032 ceph-mon[29763]: pgmap v752: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T19:00:29.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 19:00:29 smithi045 ceph-mon[33299]: pgmap v753: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T19:00:29.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 19:00:29 smithi113 ceph-mon[33540]: pgmap v753: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T19:00:29.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:00:29 smithi032 ceph-mon[29763]: pgmap v753: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T19:00:31.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 19:00:31 smithi045 ceph-mon[33299]: pgmap v754: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T19:00:31.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 19:00:31 smithi113 ceph-mon[33540]: pgmap v754: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T19:00:31.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:00:31 smithi032 ceph-mon[29763]: pgmap v754: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T19:00:33.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 19:00:33 smithi045 ceph-mon[33299]: pgmap v755: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T19:00:33.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 19:00:33 smithi113 ceph-mon[33540]: pgmap v755: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T19:00:33.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:00:33 smithi032 ceph-mon[29763]: pgmap v755: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T19:00:35.620 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:00:35 smithi032 ceph-mon[29763]: pgmap v756: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T19:00:35.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 19:00:35 smithi045 ceph-mon[33299]: pgmap v756: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T19:00:35.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 19:00:35 smithi113 ceph-mon[33540]: pgmap v756: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T19:00:37.609 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 19:00:37 smithi113 ceph-mon[33540]: pgmap v757: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T19:00:37.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 19:00:37 smithi045 ceph-mon[33299]: pgmap v757: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T19:00:37.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:00:37 smithi032 ceph-mon[29763]: pgmap v757: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T19:00:39.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 19:00:39 smithi045 ceph-mon[33299]: pgmap v758: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T19:00:39.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 19:00:39 smithi113 ceph-mon[33540]: pgmap v758: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T19:00:39.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:00:39 smithi032 ceph-mon[29763]: pgmap v758: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T19:00:41.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 19:00:41 smithi045 ceph-mon[33299]: pgmap v759: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T19:00:41.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 19:00:41 smithi113 ceph-mon[33540]: pgmap v759: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T19:00:41.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:00:41 smithi032 ceph-mon[29763]: pgmap v759: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T19:00:43.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 19:00:43 smithi045 ceph-mon[33299]: pgmap v760: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T19:00:43.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 19:00:43 smithi113 ceph-mon[33540]: pgmap v760: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T19:00:43.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:00:43 smithi032 ceph-mon[29763]: pgmap v760: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T19:00:45.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 19:00:45 smithi045 ceph-mon[33299]: pgmap v761: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T19:00:45.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 19:00:45 smithi113 ceph-mon[33540]: pgmap v761: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T19:00:45.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:00:45 smithi032 ceph-mon[29763]: pgmap v761: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T19:00:47.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 19:00:47 smithi045 ceph-mon[33299]: pgmap v762: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T19:00:47.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 19:00:47 smithi113 ceph-mon[33540]: pgmap v762: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T19:00:47.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:00:47 smithi032 ceph-mon[29763]: pgmap v762: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T19:00:49.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 19:00:49 smithi045 ceph-mon[33299]: pgmap v763: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T19:00:49.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 19:00:49 smithi113 ceph-mon[33540]: pgmap v763: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T19:00:49.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:00:49 smithi032 ceph-mon[29763]: pgmap v763: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T19:00:51.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 19:00:51 smithi045 ceph-mon[33299]: pgmap v764: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T19:00:51.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 19:00:51 smithi113 ceph-mon[33540]: pgmap v764: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T19:00:51.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:00:51 smithi032 ceph-mon[29763]: pgmap v764: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T19:00:53.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 19:00:53 smithi045 ceph-mon[33299]: pgmap v765: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T19:00:53.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 19:00:53 smithi113 ceph-mon[33540]: pgmap v765: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T19:00:53.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:00:53 smithi032 ceph-mon[29763]: pgmap v765: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T19:00:55.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 19:00:55 smithi045 ceph-mon[33299]: pgmap v766: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T19:00:55.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 19:00:55 smithi113 ceph-mon[33540]: pgmap v766: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T19:00:55.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:00:55 smithi032 ceph-mon[29763]: pgmap v766: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T19:00:57.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 19:00:57 smithi045 ceph-mon[33299]: pgmap v767: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T19:00:57.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 19:00:57 smithi113 ceph-mon[33540]: pgmap v767: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T19:00:57.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:00:57 smithi032 ceph-mon[29763]: pgmap v767: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T19:00:59.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 19:00:59 smithi045 ceph-mon[33299]: pgmap v768: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T19:00:59.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 19:00:59 smithi113 ceph-mon[33540]: pgmap v768: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T19:00:59.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:00:59 smithi032 ceph-mon[29763]: pgmap v768: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T19:01:01.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 19:01:01 smithi045 ceph-mon[33299]: pgmap v769: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T19:01:01.785 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:01:01 smithi032 ceph-mon[29763]: pgmap v769: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T19:01:01.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 19:01:01 smithi113 ceph-mon[33540]: pgmap v769: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T19:01:03.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 19:01:03 smithi045 ceph-mon[33299]: pgmap v770: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T19:01:03.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 19:01:03 smithi113 ceph-mon[33540]: pgmap v770: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T19:01:03.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:01:03 smithi032 ceph-mon[29763]: pgmap v770: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T19:01:05.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 19:01:05 smithi045 ceph-mon[33299]: pgmap v771: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T19:01:05.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 19:01:05 smithi113 ceph-mon[33540]: pgmap v771: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T19:01:05.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:01:05 smithi032 ceph-mon[29763]: pgmap v771: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T19:01:07.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 19:01:07 smithi045 ceph-mon[33299]: pgmap v772: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T19:01:07.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 19:01:07 smithi113 ceph-mon[33540]: pgmap v772: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T19:01:07.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:01:07 smithi032 ceph-mon[29763]: pgmap v772: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T19:01:09.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 19:01:09 smithi045 ceph-mon[33299]: pgmap v773: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T19:01:09.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 19:01:09 smithi113 ceph-mon[33540]: pgmap v773: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T19:01:09.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:01:09 smithi032 ceph-mon[29763]: pgmap v773: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T19:01:11.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 19:01:11 smithi045 ceph-mon[33299]: pgmap v774: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T19:01:11.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 19:01:11 smithi113 ceph-mon[33540]: pgmap v774: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T19:01:11.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:01:11 smithi032 ceph-mon[29763]: pgmap v774: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T19:01:13.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 19:01:13 smithi045 ceph-mon[33299]: pgmap v775: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T19:01:13.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 19:01:13 smithi113 ceph-mon[33540]: pgmap v775: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T19:01:13.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:01:13 smithi032 ceph-mon[29763]: pgmap v775: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T19:01:15.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 19:01:15 smithi045 ceph-mon[33299]: pgmap v776: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T19:01:15.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 19:01:15 smithi113 ceph-mon[33540]: pgmap v776: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T19:01:15.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:01:15 smithi032 ceph-mon[29763]: pgmap v776: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T19:01:17.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 19:01:17 smithi045 ceph-mon[33299]: pgmap v777: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T19:01:17.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 19:01:17 smithi113 ceph-mon[33540]: pgmap v777: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T19:01:17.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:01:17 smithi032 ceph-mon[29763]: pgmap v777: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T19:01:19.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 19:01:19 smithi045 ceph-mon[33299]: pgmap v778: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T19:01:19.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 19:01:19 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T19:01:19.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 19:01:19 smithi113 ceph-mon[33540]: pgmap v778: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T19:01:19.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 19:01:19 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T19:01:19.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:01:19 smithi032 ceph-mon[29763]: pgmap v778: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T19:01:19.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:01:19 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T19:01:20.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 19:01:20 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T19:01:20.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 19:01:20 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T19:01:20.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 19:01:20 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T19:01:20.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 19:01:20 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T19:01:20.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 19:01:20 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T19:01:20.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 19:01:20 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T19:01:20.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:01:20 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T19:01:20.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:01:20 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T19:01:20.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:01:20 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T19:01:21.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 19:01:21 smithi045 ceph-mon[33299]: pgmap v779: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T19:01:21.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 19:01:21 smithi113 ceph-mon[33540]: pgmap v779: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T19:01:21.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:01:21 smithi032 ceph-mon[29763]: pgmap v779: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T19:01:23.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 19:01:23 smithi045 ceph-mon[33299]: pgmap v780: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T19:01:23.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 19:01:23 smithi113 ceph-mon[33540]: pgmap v780: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T19:01:23.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:01:23 smithi032 ceph-mon[29763]: pgmap v780: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T19:01:25.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 19:01:25 smithi045 ceph-mon[33299]: pgmap v781: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T19:01:25.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 19:01:25 smithi113 ceph-mon[33540]: pgmap v781: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T19:01:25.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:01:25 smithi032 ceph-mon[29763]: pgmap v781: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T19:01:27.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 19:01:27 smithi045 ceph-mon[33299]: pgmap v782: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T19:01:27.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 19:01:27 smithi113 ceph-mon[33540]: pgmap v782: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T19:01:27.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:01:27 smithi032 ceph-mon[29763]: pgmap v782: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T19:01:29.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 19:01:29 smithi045 ceph-mon[33299]: pgmap v783: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T19:01:29.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 19:01:29 smithi113 ceph-mon[33540]: pgmap v783: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T19:01:29.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:01:29 smithi032 ceph-mon[29763]: pgmap v783: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T19:01:31.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 19:01:31 smithi045 ceph-mon[33299]: pgmap v784: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T19:01:31.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 19:01:31 smithi113 ceph-mon[33540]: pgmap v784: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T19:01:31.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:01:31 smithi032 ceph-mon[29763]: pgmap v784: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T19:01:33.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 19:01:33 smithi045 ceph-mon[33299]: pgmap v785: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T19:01:33.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 19:01:33 smithi113 ceph-mon[33540]: pgmap v785: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T19:01:33.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:01:33 smithi032 ceph-mon[29763]: pgmap v785: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T19:01:35.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 19:01:35 smithi045 ceph-mon[33299]: pgmap v786: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T19:01:35.802 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 19:01:35 smithi113 ceph-mon[33540]: pgmap v786: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T19:01:35.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:01:35 smithi032 ceph-mon[29763]: pgmap v786: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T19:01:37.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 19:01:37 smithi045 ceph-mon[33299]: pgmap v787: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T19:01:37.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 19:01:37 smithi113 ceph-mon[33540]: pgmap v787: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T19:01:37.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:01:37 smithi032 ceph-mon[29763]: pgmap v787: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T19:01:39.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 19:01:39 smithi045 ceph-mon[33299]: pgmap v788: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T19:01:39.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 19:01:39 smithi113 ceph-mon[33540]: pgmap v788: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T19:01:39.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:01:39 smithi032 ceph-mon[29763]: pgmap v788: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T19:01:41.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 19:01:41 smithi045 ceph-mon[33299]: pgmap v789: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T19:01:41.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 19:01:41 smithi113 ceph-mon[33540]: pgmap v789: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T19:01:41.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:01:41 smithi032 ceph-mon[29763]: pgmap v789: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T19:01:43.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 19:01:43 smithi045 ceph-mon[33299]: pgmap v790: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T19:01:43.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 19:01:43 smithi113 ceph-mon[33540]: pgmap v790: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T19:01:43.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:01:43 smithi032 ceph-mon[29763]: pgmap v790: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T19:01:45.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 19:01:45 smithi045 ceph-mon[33299]: pgmap v791: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T19:01:45.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 19:01:45 smithi113 ceph-mon[33540]: pgmap v791: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T19:01:45.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:01:45 smithi032 ceph-mon[29763]: pgmap v791: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T19:01:47.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 19:01:47 smithi045 ceph-mon[33299]: pgmap v792: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T19:01:47.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 19:01:47 smithi113 ceph-mon[33540]: pgmap v792: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T19:01:47.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:01:47 smithi032 ceph-mon[29763]: pgmap v792: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T19:01:49.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 19:01:49 smithi045 ceph-mon[33299]: pgmap v793: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T19:01:49.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 19:01:49 smithi113 ceph-mon[33540]: pgmap v793: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T19:01:49.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:01:49 smithi032 ceph-mon[29763]: pgmap v793: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T19:01:51.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 19:01:51 smithi045 ceph-mon[33299]: pgmap v794: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T19:01:51.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 19:01:51 smithi113 ceph-mon[33540]: pgmap v794: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T19:01:51.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:01:51 smithi032 ceph-mon[29763]: pgmap v794: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T19:01:53.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 19:01:53 smithi045 ceph-mon[33299]: pgmap v795: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T19:01:53.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 19:01:53 smithi113 ceph-mon[33540]: pgmap v795: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T19:01:53.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:01:53 smithi032 ceph-mon[29763]: pgmap v795: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T19:01:55.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 19:01:55 smithi045 ceph-mon[33299]: pgmap v796: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T19:01:55.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 19:01:55 smithi113 ceph-mon[33540]: pgmap v796: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T19:01:55.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:01:55 smithi032 ceph-mon[29763]: pgmap v796: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T19:01:57.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 19:01:57 smithi045 ceph-mon[33299]: pgmap v797: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T19:01:57.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 19:01:57 smithi113 ceph-mon[33540]: pgmap v797: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T19:01:57.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:01:57 smithi032 ceph-mon[29763]: pgmap v797: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T19:01:59.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 19:01:59 smithi045 ceph-mon[33299]: pgmap v798: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T19:01:59.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 19:01:59 smithi113 ceph-mon[33540]: pgmap v798: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T19:01:59.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:01:59 smithi032 ceph-mon[29763]: pgmap v798: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T19:02:01.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 19:02:01 smithi045 ceph-mon[33299]: pgmap v799: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T19:02:01.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 19:02:01 smithi113 ceph-mon[33540]: pgmap v799: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T19:02:01.819 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:02:01 smithi032 ceph-mon[29763]: pgmap v799: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T19:02:03.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 19:02:03 smithi045 ceph-mon[33299]: pgmap v800: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T19:02:03.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 19:02:03 smithi113 ceph-mon[33540]: pgmap v800: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T19:02:03.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:02:03 smithi032 ceph-mon[29763]: pgmap v800: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T19:02:05.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 19:02:05 smithi045 ceph-mon[33299]: pgmap v801: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T19:02:05.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 19:02:05 smithi113 ceph-mon[33540]: pgmap v801: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T19:02:05.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:02:05 smithi032 ceph-mon[29763]: pgmap v801: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T19:02:07.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 19:02:07 smithi045 ceph-mon[33299]: pgmap v802: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T19:02:07.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 19:02:07 smithi113 ceph-mon[33540]: pgmap v802: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T19:02:07.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:02:07 smithi032 ceph-mon[29763]: pgmap v802: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T19:02:09.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 19:02:09 smithi045 ceph-mon[33299]: pgmap v803: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T19:02:09.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 19:02:09 smithi113 ceph-mon[33540]: pgmap v803: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T19:02:09.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:02:09 smithi032 ceph-mon[29763]: pgmap v803: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T19:02:11.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 19:02:11 smithi045 ceph-mon[33299]: pgmap v804: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T19:02:11.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 19:02:11 smithi113 ceph-mon[33540]: pgmap v804: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T19:02:11.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:02:11 smithi032 ceph-mon[29763]: pgmap v804: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T19:02:13.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 19:02:13 smithi045 ceph-mon[33299]: pgmap v805: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T19:02:13.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 19:02:13 smithi113 ceph-mon[33540]: pgmap v805: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T19:02:13.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:02:13 smithi032 ceph-mon[29763]: pgmap v805: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T19:02:15.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 19:02:15 smithi045 ceph-mon[33299]: pgmap v806: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T19:02:15.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 19:02:15 smithi113 ceph-mon[33540]: pgmap v806: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T19:02:15.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:02:15 smithi032 ceph-mon[29763]: pgmap v806: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T19:02:17.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 19:02:17 smithi045 ceph-mon[33299]: pgmap v807: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T19:02:17.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 19:02:17 smithi113 ceph-mon[33540]: pgmap v807: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T19:02:17.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:02:17 smithi032 ceph-mon[29763]: pgmap v807: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T19:02:19.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 19:02:19 smithi045 ceph-mon[33299]: pgmap v808: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T19:02:19.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 19:02:19 smithi113 ceph-mon[33540]: pgmap v808: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T19:02:19.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:02:19 smithi032 ceph-mon[29763]: pgmap v808: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T19:02:20.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 19:02:20 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T19:02:20.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:02:20 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T19:02:20.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 19:02:20 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-03T19:02:21.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 19:02:21 smithi113 ceph-mon[33540]: pgmap v809: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T19:02:21.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 19:02:21 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T19:02:21.802 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 19:02:21 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T19:02:21.802 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 19:02:21 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T19:02:21.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:02:21 smithi032 ceph-mon[29763]: pgmap v809: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T19:02:21.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:02:21 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T19:02:21.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:02:21 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T19:02:21.819 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:02:21 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T19:02:21.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 19:02:21 smithi045 ceph-mon[33299]: pgmap v809: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T19:02:21.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 19:02:21 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-03T19:02:21.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 19:02:21 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-03T19:02:21.999 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 19:02:21 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' 2024-05-03T19:02:23.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 19:02:23 smithi113 ceph-mon[33540]: pgmap v810: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T19:02:23.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:02:23 smithi032 ceph-mon[29763]: pgmap v810: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T19:02:23.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 19:02:23 smithi045 ceph-mon[33299]: pgmap v810: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T19:02:25.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 19:02:25 smithi113 ceph-mon[33540]: pgmap v811: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T19:02:25.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:02:25 smithi032 ceph-mon[29763]: pgmap v811: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T19:02:25.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 19:02:25 smithi045 ceph-mon[33299]: pgmap v811: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T19:02:27.801 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 19:02:27 smithi045 ceph-mon[33299]: pgmap v812: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T19:02:27.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 19:02:27 smithi113 ceph-mon[33540]: pgmap v812: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T19:02:27.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:02:27 smithi032 ceph-mon[29763]: pgmap v812: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T19:02:29.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 19:02:29 smithi113 ceph-mon[33540]: pgmap v813: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T19:02:29.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:02:29 smithi032 ceph-mon[29763]: pgmap v813: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T19:02:29.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 19:02:29 smithi045 ceph-mon[33299]: pgmap v813: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T19:02:31.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 19:02:31 smithi113 ceph-mon[33540]: pgmap v814: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T19:02:31.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:02:31 smithi032 ceph-mon[29763]: pgmap v814: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T19:02:31.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 19:02:31 smithi045 ceph-mon[33299]: pgmap v814: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T19:02:33.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 19:02:33 smithi113 ceph-mon[33540]: pgmap v815: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T19:02:33.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:02:33 smithi032 ceph-mon[29763]: pgmap v815: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T19:02:33.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 19:02:33 smithi045 ceph-mon[33299]: pgmap v815: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T19:02:35.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 19:02:35 smithi113 ceph-mon[33540]: pgmap v816: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T19:02:35.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:02:35 smithi032 ceph-mon[29763]: pgmap v816: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T19:02:35.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 19:02:35 smithi045 ceph-mon[33299]: pgmap v816: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T19:02:37.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 19:02:37 smithi113 ceph-mon[33540]: pgmap v817: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T19:02:37.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:02:37 smithi032 ceph-mon[29763]: pgmap v817: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T19:02:37.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 19:02:37 smithi045 ceph-mon[33299]: pgmap v817: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T19:02:39.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 19:02:39 smithi113 ceph-mon[33540]: pgmap v818: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T19:02:39.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:02:39 smithi032 ceph-mon[29763]: pgmap v818: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T19:02:39.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 19:02:39 smithi045 ceph-mon[33299]: pgmap v818: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T19:02:41.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 19:02:41 smithi113 ceph-mon[33540]: pgmap v819: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T19:02:41.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:02:41 smithi032 ceph-mon[29763]: pgmap v819: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T19:02:41.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 19:02:41 smithi045 ceph-mon[33299]: pgmap v819: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T19:02:43.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 19:02:43 smithi113 ceph-mon[33540]: pgmap v820: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T19:02:43.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:02:43 smithi032 ceph-mon[29763]: pgmap v820: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T19:02:43.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 19:02:43 smithi045 ceph-mon[33299]: pgmap v820: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T19:02:45.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 19:02:45 smithi113 ceph-mon[33540]: pgmap v821: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T19:02:45.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:02:45 smithi032 ceph-mon[29763]: pgmap v821: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T19:02:45.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 19:02:45 smithi045 ceph-mon[33299]: pgmap v821: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T19:02:47.492 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:47.491+0000 7f9a1e59c640 1 -- 172.21.15.32:0/3388640237 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f9a080b2920 msgr2=0x7f9a080b4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).tick idle (900026517) for more than 900000000 us, fault. 2024-05-03T19:02:47.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:47.491+0000 7f9a1e59c640 1 --2- 172.21.15.32:0/3388640237 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f9a080b2920 0x7f9a080b4de0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f9a10000f80 tx=0x7f9a10005ef0 comp rx=0 tx=0).stop 2024-05-03T19:02:47.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:47.491+0000 7f9a0f7fe640 1 -- 172.21.15.32:0/3388640237 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f9a080b2920 msgr2=0x7f9a080b4de0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-03T19:02:47.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:47.491+0000 7f9a0f7fe640 1 --2- 172.21.15.32:0/3388640237 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f9a080b2920 0x7f9a080b4de0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T19:02:47.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:47.491+0000 7f9a0f7fe640 1 --2- 172.21.15.32:0/3388640237 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f9a080bcbb0 0x7f9a080befa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T19:02:47.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:47.491+0000 7f9a0f7fe640 1 -- 172.21.15.32:0/3388640237 --> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7f9a14005320 con 0x7f9a080bcbb0 2024-05-03T19:02:47.493 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:47.491+0000 7f9a1e59c640 1 --2- 172.21.15.32:0/3388640237 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f9a080bcbb0 0x7f9a080befa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T19:02:47.494 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:47.492+0000 7f9a1e59c640 1 --2- 172.21.15.32:0/3388640237 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f9a080bcbb0 0x7f9a080befa0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f9a10000f80 tx=0x7f9a10041000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T19:02:47.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:47.589+0000 7f9a0e7fc640 1 -- 172.21.15.32:0/3388640237 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7f9a04006350 con 0x7f9a20102b00 2024-05-03T19:02:47.590 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:47.589+0000 7f9a0f7fe640 1 -- 172.21.15.32:0/3388640237 <== mon.0 v2:172.21.15.32:3300/0 12 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7f9a04006350 con 0x7f9a20102b00 2024-05-03T19:02:47.691 INFO:teuthology.orchestra.run.smithi032.stderr:Error ETIMEDOUT: Connection timed out. Please check the client's permission and connection. 2024-05-03T19:02:47.698 DEBUG:teuthology.orchestra.run:got remote process result: 110 2024-05-03T19:02:47.698 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/task/exec.py", line 66, in task remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi032 with status 110: "sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10'" 2024-05-03T19:02:47.940 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=fd182ab35b5649f18da4ca8d5ff715b4 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/task/exec.py", line 66, in task remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi032 with status 110: "sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10'" 2024-05-03T19:02:47.942 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-05-03T19:02:47.955 INFO:tasks.kclient:Unmounting kernel clients... 2024-05-03T19:02:47.956 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-05-03T19:02:47.956 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-03T19:02:47.960 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 19:02:47 smithi113 ceph-mon[33540]: pgmap v822: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T19:02:47.960 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:02:47 smithi032 ceph-mon[29763]: pgmap v822: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T19:02:47.980 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2024-05-03T19:02:47.981 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-03T19:02:47.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 19:02:47 smithi045 ceph-mon[33299]: pgmap v822: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T19:02:48.041 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid f500455e-097b-11ef-bc97-c7b262605968 -- ceph osd blocklist ls 2024-05-03T19:02:48.338 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T19:02:48.801 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:02:48 smithi032 ceph-mon[29763]: from='client.15150 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T19:02:48.802 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 19:02:48 smithi113 ceph-mon[33540]: from='client.15150 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T19:02:48.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 19:02:48 smithi045 ceph-mon[33299]: from='client.15150 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-05-03T19:02:49.615 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:02:49 smithi032 ceph-mon[29763]: pgmap v823: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T19:02:49.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 19:02:49 smithi113 ceph-mon[33540]: pgmap v823: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T19:02:49.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 19:02:49 smithi045 ceph-mon[33299]: pgmap v823: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-05-03T19:02:50.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:49.910+0000 7f6e09774700 1 -- 172.21.15.32:0/3390099439 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e04103750 msgr2=0x7f6e04103b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T19:02:50.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:49.910+0000 7f6e09774700 1 --2- 172.21.15.32:0/3390099439 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e04103750 0x7f6e04103b30 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f6df40099c0 tx=0x7f6df4009cd0 comp rx=0 tx=0).stop 2024-05-03T19:02:50.060 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:49.910+0000 7f6e09774700 1 -- 172.21.15.32:0/3390099439 shutdown_connections 2024-05-03T19:02:50.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:49.910+0000 7f6e09774700 1 --2- 172.21.15.32:0/3390099439 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e04108850 0x7f6e041103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T19:02:50.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:49.910+0000 7f6e09774700 1 --2- 172.21.15.32:0/3390099439 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e04103750 0x7f6e04103b30 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T19:02:50.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:49.910+0000 7f6e09774700 1 --2- 172.21.15.32:0/3390099439 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6e04104100 0x7f6e04108120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T19:02:50.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:49.910+0000 7f6e09774700 1 -- 172.21.15.32:0/3390099439 >> 172.21.15.32:0/3390099439 conn(0x7f6e0407a600 msgr2=0x7f6e0407aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T19:02:50.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:49.910+0000 7f6e09774700 1 -- 172.21.15.32:0/3390099439 shutdown_connections 2024-05-03T19:02:50.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:49.910+0000 7f6e09774700 1 -- 172.21.15.32:0/3390099439 wait complete. 2024-05-03T19:02:50.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:49.910+0000 7f6e09774700 1 Processor -- start 2024-05-03T19:02:50.061 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:49.911+0000 7f6e09774700 1 -- start start 2024-05-03T19:02:50.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:49.911+0000 7f6e09774700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6e04103750 0x7f6e040757a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T19:02:50.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:49.911+0000 7f6e09774700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e04104100 0x7f6e04075ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T19:02:50.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:49.911+0000 7f6e09774700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e04108850 0x7f6e04076220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T19:02:50.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:49.911+0000 7f6e09774700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f6e04113760 con 0x7f6e04108850 2024-05-03T19:02:50.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:49.911+0000 7f6e09774700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6e041135e0 con 0x7f6e04104100 2024-05-03T19:02:50.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:49.911+0000 7f6e09774700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f6e041138e0 con 0x7f6e04103750 2024-05-03T19:02:50.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:49.911+0000 7f6e037fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e04108850 0x7f6e04076220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T19:02:50.062 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:49.911+0000 7f6e037fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e04108850 0x7f6e04076220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57076/0 (socket says 172.21.15.32:57076) 2024-05-03T19:02:50.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:49.911+0000 7f6e037fe700 1 -- 172.21.15.32:0/3890236478 learned_addr learned my addr 172.21.15.32:0/3890236478 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T19:02:50.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:49.911+0000 7f6e027fc700 1 --2- 172.21.15.32:0/3890236478 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e04104100 0x7f6e04075ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T19:02:50.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:49.911+0000 7f6e02ffd700 1 --2- 172.21.15.32:0/3890236478 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6e04103750 0x7f6e040757a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T19:02:50.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:49.912+0000 7f6e037fe700 1 -- 172.21.15.32:0/3890236478 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6e04103750 msgr2=0x7f6e040757a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T19:02:50.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:49.912+0000 7f6e037fe700 1 --2- 172.21.15.32:0/3890236478 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6e04103750 0x7f6e040757a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T19:02:50.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:49.912+0000 7f6e037fe700 1 -- 172.21.15.32:0/3890236478 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e04104100 msgr2=0x7f6e04075ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T19:02:50.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:49.912+0000 7f6e037fe700 1 --2- 172.21.15.32:0/3890236478 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e04104100 0x7f6e04075ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T19:02:50.063 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:49.912+0000 7f6e037fe700 1 -- 172.21.15.32:0/3890236478 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e041a8160 con 0x7f6e04108850 2024-05-03T19:02:50.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:49.912+0000 7f6e037fe700 1 --2- 172.21.15.32:0/3890236478 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e04108850 0x7f6e04076220 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f6dfc00b970 tx=0x7f6dfc00bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T19:02:50.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:49.912+0000 7f6debfff700 1 -- 172.21.15.32:0/3890236478 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dfc00d860 con 0x7f6e04108850 2024-05-03T19:02:50.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:49.912+0000 7f6e09774700 1 -- 172.21.15.32:0/3890236478 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e041a8450 con 0x7f6e04108850 2024-05-03T19:02:50.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:49.912+0000 7f6debfff700 1 -- 172.21.15.32:0/3890236478 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6dfc00e430 con 0x7f6e04108850 2024-05-03T19:02:50.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:49.912+0000 7f6debfff700 1 -- 172.21.15.32:0/3890236478 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dfc017660 con 0x7f6e04108850 2024-05-03T19:02:50.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:49.912+0000 7f6e09774700 1 -- 172.21.15.32:0/3890236478 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e041a89e0 con 0x7f6e04108850 2024-05-03T19:02:50.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:49.914+0000 7f6e09774700 1 -- 172.21.15.32:0/3890236478 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e0410ba90 con 0x7f6e04108850 2024-05-03T19:02:50.064 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:49.914+0000 7f6debfff700 1 -- 172.21.15.32:0/3890236478 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f6dfc00da00 con 0x7f6e04108850 2024-05-03T19:02:50.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:49.915+0000 7f6debfff700 1 --2- 172.21.15.32:0/3890236478 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f6dec0b2910 0x7f6dec0b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T19:02:50.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:49.915+0000 7f6e02ffd700 1 --2- 172.21.15.32:0/3890236478 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f6dec0b2910 0x7f6dec0b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T19:02:50.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:49.915+0000 7f6debfff700 1 -- 172.21.15.32:0/3890236478 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8356+0+0 (secure 0 0 0) 0x7f6dfc0e1fc0 con 0x7f6e04108850 2024-05-03T19:02:50.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:49.916+0000 7f6e02ffd700 1 --2- 172.21.15.32:0/3890236478 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f6dec0b2910 0x7f6dec0b4dd0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f6df400b580 tx=0x7f6df40096d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T19:02:50.065 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:49.920+0000 7f6debfff700 1 -- 172.21.15.32:0/3890236478 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f6dfc013030 con 0x7f6e04108850 2024-05-03T19:02:50.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:50.153+0000 7f6e09774700 1 -- 172.21.15.32:0/3890236478 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f6e04076cd0 con 0x7f6e04108850 2024-05-03T19:02:50.155 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:50.154+0000 7f6debfff700 1 -- 172.21.15.32:0/3890236478 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v82) ==== 81+0+591 (secure 0 0 0) 0x7f6dfc0a9b00 con 0x7f6e04108850 2024-05-03T19:02:50.155 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/743758274 2024-05-04T18:37:45.879602+0000 2024-05-03T19:02:50.156 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:6801/2019702098 2024-05-04T18:37:45.879602+0000 2024-05-03T19:02:50.156 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:6800/2019702098 2024-05-04T18:37:45.879602+0000 2024-05-03T19:02:50.156 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/3192385439 2024-05-04T18:37:18.492319+0000 2024-05-03T19:02:50.156 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/1450308494 2024-05-04T18:37:18.492319+0000 2024-05-03T19:02:50.156 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/3957699240 2024-05-04T18:37:45.879602+0000 2024-05-03T19:02:50.156 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/3310503393 2024-05-04T18:37:18.492319+0000 2024-05-03T19:02:50.156 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:6801/3945405684 2024-05-04T18:37:18.492319+0000 2024-05-03T19:02:50.157 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:6800/3945405684 2024-05-04T18:37:18.492319+0000 2024-05-03T19:02:50.157 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/2471164468 2024-05-04T18:37:45.879602+0000 2024-05-03T19:02:50.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:50.156+0000 7f6e09774700 1 -- 172.21.15.32:0/3890236478 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f6dec0b2910 msgr2=0x7f6dec0b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T19:02:50.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:50.156+0000 7f6e09774700 1 --2- 172.21.15.32:0/3890236478 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f6dec0b2910 0x7f6dec0b4dd0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f6df400b580 tx=0x7f6df40096d0 comp rx=0 tx=0).stop 2024-05-03T19:02:50.158 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:50.156+0000 7f6e09774700 1 -- 172.21.15.32:0/3890236478 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e04108850 msgr2=0x7f6e04076220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T19:02:50.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:50.156+0000 7f6e09774700 1 --2- 172.21.15.32:0/3890236478 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e04108850 0x7f6e04076220 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f6dfc00b970 tx=0x7f6dfc00bc80 comp rx=0 tx=0).stop 2024-05-03T19:02:50.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:50.157+0000 7f6e09774700 1 -- 172.21.15.32:0/3890236478 shutdown_connections 2024-05-03T19:02:50.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:50.157+0000 7f6e09774700 1 --2- 172.21.15.32:0/3890236478 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f6dec0b2910 0x7f6dec0b4dd0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T19:02:50.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:50.157+0000 7f6e09774700 1 --2- 172.21.15.32:0/3890236478 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e04108850 0x7f6e04076220 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T19:02:50.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:50.157+0000 7f6e09774700 1 --2- 172.21.15.32:0/3890236478 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6e04103750 0x7f6e040757a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T19:02:50.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:50.157+0000 7f6e09774700 1 --2- 172.21.15.32:0/3890236478 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e04104100 0x7f6e04075ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T19:02:50.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:50.157+0000 7f6e09774700 1 -- 172.21.15.32:0/3890236478 >> 172.21.15.32:0/3890236478 conn(0x7f6e0407a600 msgr2=0x7f6e04101710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T19:02:50.159 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:50.157+0000 7f6e09774700 1 -- 172.21.15.32:0/3890236478 shutdown_connections 2024-05-03T19:02:50.160 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:50.157+0000 7f6e09774700 1 -- 172.21.15.32:0/3890236478 wait complete. 2024-05-03T19:02:50.160 INFO:teuthology.orchestra.run.smithi032.stderr:listed 10 entries 2024-05-03T19:02:50.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:02:50 smithi032 ceph-mon[29763]: from='client.? 172.21.15.32:0/3890236478' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-03T19:02:50.907 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-03T19:02:50.907 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-03T19:02:50.907 DEBUG:teuthology.orchestra.run.smithi032:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-05-03T19:02:50.943 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi032.front.sepia.ceph.com 2024-05-03T19:02:50.943 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-03T19:02:50.943 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-03T19:02:50.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 19:02:50 smithi045 ceph-mon[33299]: from='client.? 172.21.15.32:0/3890236478' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-03T19:02:51.002 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-03T19:02:51.003 DEBUG:teuthology.orchestra.run.smithi032:> ip netns list 2024-05-03T19:02:51.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 19:02:50 smithi113 ceph-mon[33540]: from='client.? 172.21.15.32:0/3890236478' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-03T19:02:51.059 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-03T19:02:51.060 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-03T19:02:51.060 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-03T19:02:51.126 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-03T19:02:51.127 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link delete ceph-brx 2024-05-03T19:02:51.226 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-05-03T19:02:51.236 INFO:tasks.cephadm:Teardown begin 2024-05-03T19:02:51.237 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-03T19:02:51.237 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid f500455e-097b-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-03T19:02:51.489 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T19:02:51.740 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:02:51 smithi032 ceph-mon[29763]: pgmap v824: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T19:02:51.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 19:02:51 smithi045 ceph-mon[33299]: pgmap v824: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T19:02:52.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 19:02:51 smithi113 ceph-mon[33540]: pgmap v824: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-03T19:02:53.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:53.164+0000 7f088fb81700 1 -- 172.21.15.32:0/3084341479 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0888102620 msgr2=0x7f0888102a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T19:02:53.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:53.164+0000 7f088fb81700 1 --2- 172.21.15.32:0/3084341479 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0888102620 0x7f0888102a00 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f08840099c0 tx=0x7f0884009cd0 comp rx=0 tx=0).stop 2024-05-03T19:02:53.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:53.164+0000 7f088fb81700 1 -- 172.21.15.32:0/3084341479 shutdown_connections 2024-05-03T19:02:53.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:53.164+0000 7f088fb81700 1 --2- 172.21.15.32:0/3084341479 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0888107700 0x7f088810f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T19:02:53.168 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:53.164+0000 7f088fb81700 1 --2- 172.21.15.32:0/3084341479 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0888102620 0x7f0888102a00 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T19:02:53.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:53.164+0000 7f088fb81700 1 --2- 172.21.15.32:0/3084341479 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0888102fd0 0x7f0888107020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T19:02:53.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:53.164+0000 7f088fb81700 1 -- 172.21.15.32:0/3084341479 >> 172.21.15.32:0/3084341479 conn(0x7f0888079440 msgr2=0x7f0888079850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T19:02:53.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:53.164+0000 7f088fb81700 1 -- 172.21.15.32:0/3084341479 shutdown_connections 2024-05-03T19:02:53.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:53.164+0000 7f088fb81700 1 -- 172.21.15.32:0/3084341479 wait complete. 2024-05-03T19:02:53.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:53.165+0000 7f088fb81700 1 Processor -- start 2024-05-03T19:02:53.169 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:53.165+0000 7f088fb81700 1 -- start start 2024-05-03T19:02:53.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:53.165+0000 7f088fb81700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0888102620 0x7f088819f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T19:02:53.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:53.165+0000 7f088fb81700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0888102fd0 0x7f088819fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T19:02:53.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:53.165+0000 7f088fb81700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0888107700 0x7f08881999f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T19:02:53.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:53.165+0000 7f088fb81700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f08881125b0 con 0x7f0888107700 2024-05-03T19:02:53.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:53.165+0000 7f088fb81700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f0888112430 con 0x7f0888102fd0 2024-05-03T19:02:53.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:53.165+0000 7f088fb81700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f0888112730 con 0x7f0888102620 2024-05-03T19:02:53.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:53.165+0000 7f088e11e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0888107700 0x7f08881999f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T19:02:53.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:53.165+0000 7f088e11e700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0888107700 0x7f08881999f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57096/0 (socket says 172.21.15.32:57096) 2024-05-03T19:02:53.170 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:53.165+0000 7f088e11e700 1 -- 172.21.15.32:0/3576033756 learned_addr learned my addr 172.21.15.32:0/3576033756 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T19:02:53.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:53.165+0000 7f088d11c700 1 --2- 172.21.15.32:0/3576033756 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0888102fd0 0x7f088819fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T19:02:53.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:53.165+0000 7f088d91d700 1 --2- 172.21.15.32:0/3576033756 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0888102620 0x7f088819f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T19:02:53.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:53.165+0000 7f088e11e700 1 -- 172.21.15.32:0/3576033756 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0888102620 msgr2=0x7f088819f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T19:02:53.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:53.165+0000 7f088e11e700 1 --2- 172.21.15.32:0/3576033756 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0888102620 0x7f088819f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T19:02:53.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:53.165+0000 7f088e11e700 1 -- 172.21.15.32:0/3576033756 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0888102fd0 msgr2=0x7f088819fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T19:02:53.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:53.165+0000 7f088e11e700 1 --2- 172.21.15.32:0/3576033756 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0888102fd0 0x7f088819fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T19:02:53.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:53.165+0000 7f088e11e700 1 -- 172.21.15.32:0/3576033756 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f088819a150 con 0x7f0888107700 2024-05-03T19:02:53.171 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:53.165+0000 7f088d91d700 1 --2- 172.21.15.32:0/3576033756 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0888102620 0x7f088819f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T19:02:53.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:53.166+0000 7f088e11e700 1 --2- 172.21.15.32:0/3576033756 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0888107700 0x7f08881999f0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f0880007ee0 tx=0x7f088000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T19:02:53.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:53.166+0000 7f087effd700 1 -- 172.21.15.32:0/3576033756 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f088000dbd0 con 0x7f0888107700 2024-05-03T19:02:53.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:53.166+0000 7f087effd700 1 -- 172.21.15.32:0/3576033756 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f088000dd70 con 0x7f0888107700 2024-05-03T19:02:53.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:53.166+0000 7f088fb81700 1 -- 172.21.15.32:0/3576033756 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f088819a440 con 0x7f0888107700 2024-05-03T19:02:53.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:53.166+0000 7f087effd700 1 -- 172.21.15.32:0/3576033756 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0880005990 con 0x7f0888107700 2024-05-03T19:02:53.172 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:53.166+0000 7f088fb81700 1 -- 172.21.15.32:0/3576033756 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08881a6700 con 0x7f0888107700 2024-05-03T19:02:53.173 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:53.167+0000 7f088fb81700 1 -- 172.21.15.32:0/3576033756 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f088806a190 con 0x7f0888107700 2024-05-03T19:02:53.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:53.168+0000 7f087effd700 1 -- 172.21.15.32:0/3576033756 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f088001a020 con 0x7f0888107700 2024-05-03T19:02:53.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:53.169+0000 7f087effd700 1 --2- 172.21.15.32:0/3576033756 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f08740b2910 0x7f08740b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T19:02:53.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:53.169+0000 7f088d91d700 1 --2- 172.21.15.32:0/3576033756 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f08740b2910 0x7f08740b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T19:02:53.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:53.169+0000 7f087effd700 1 -- 172.21.15.32:0/3576033756 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8356+0+0 (secure 0 0 0) 0x7f08800e2260 con 0x7f0888107700 2024-05-03T19:02:53.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:53.169+0000 7f088d91d700 1 --2- 172.21.15.32:0/3576033756 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f08740b2910 0x7f08740b4dd0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f0884000c00 tx=0x7f0884005940 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T19:02:53.174 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:53.172+0000 7f087effd700 1 -- 172.21.15.32:0/3576033756 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f0880010070 con 0x7f0888107700 2024-05-03T19:02:53.440 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:53.439+0000 7f088fb81700 1 -- 172.21.15.32:0/3576033756 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f088802e510 con 0x7f0888107700 2024-05-03T19:02:53.441 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:53.440+0000 7f087effd700 1 -- 172.21.15.32:0/3576033756 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11862 (secure 0 0 0) 0x7f08800a9d40 con 0x7f0888107700 2024-05-03T19:02:53.441 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-03T19:02:53.442 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/535599929","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":535599929},{"type":"v1","addr":"172.21.15.32:6835","nonce":535599929}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/3101679868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":3101679868},{"type":"v1","addr":"172.21.15.32:6837","nonce":3101679868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14610,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/3838653921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":3838653921},{"type":"v1","addr":"172.21.15.32:6839","nonce":3838653921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14622,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6841/3019648436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6840","nonce":3019648436},{"type":"v1","addr":"172.21.15.32:6841","nonce":3019648436}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24380,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/1629869435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":1629869435},{"type":"v1","addr":"172.21.15.45:6833","nonce":1629869435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24398,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2997291166","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2997291166},{"type":"v1","addr":"172.21.15.45:6835","nonce":2997291166}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-03T18:44:51.155050+0000","modified":"2024-05-03T18:47:46.959629+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24478,"mds_1":24469,"mds_2":24463},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24425":{"gid":24425,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.45:6837/428055392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":428055392},{"type":"v1","addr":"172.21.15.45:6837","nonce":428055392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.113:6833/3323558791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":3323558791},{"type":"v1","addr":"172.21.15.113:6833","nonce":3323558791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"b","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.45:6839/970804040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":970804040},{"type":"v1","addr":"172.21.15.45:6839","nonce":970804040}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.113:6835/2588738276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2588738276},{"type":"v1","addr":"172.21.15.113:6835","nonce":2588738276}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.113:6837/3677284460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":3677284460},{"type":"v1","addr":"172.21.15.113:6837","nonce":3677284460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.113:6839/942050693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6838","nonce":942050693},{"type":"v1","addr":"172.21.15.113:6839","nonce":942050693}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24478,"qdb_cluster":[24463,24469,24478]},"id":1}]} 2024-05-03T19:02:53.443 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:53.442+0000 7f088fb81700 1 -- 172.21.15.32:0/3576033756 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f08740b2910 msgr2=0x7f08740b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T19:02:53.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:53.442+0000 7f088fb81700 1 --2- 172.21.15.32:0/3576033756 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f08740b2910 0x7f08740b4dd0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f0884000c00 tx=0x7f0884005940 comp rx=0 tx=0).stop 2024-05-03T19:02:53.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:53.442+0000 7f088fb81700 1 -- 172.21.15.32:0/3576033756 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0888107700 msgr2=0x7f08881999f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T19:02:53.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:53.442+0000 7f088fb81700 1 --2- 172.21.15.32:0/3576033756 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0888107700 0x7f08881999f0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f0880007ee0 tx=0x7f088000d3b0 comp rx=0 tx=0).stop 2024-05-03T19:02:53.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:53.442+0000 7f088fb81700 1 -- 172.21.15.32:0/3576033756 shutdown_connections 2024-05-03T19:02:53.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:53.442+0000 7f088fb81700 1 --2- 172.21.15.32:0/3576033756 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f08740b2910 0x7f08740b4dd0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T19:02:53.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:53.442+0000 7f088fb81700 1 --2- 172.21.15.32:0/3576033756 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0888107700 0x7f08881999f0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T19:02:53.444 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:53.442+0000 7f088fb81700 1 --2- 172.21.15.32:0/3576033756 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0888102620 0x7f088819f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T19:02:53.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:53.442+0000 7f088fb81700 1 --2- 172.21.15.32:0/3576033756 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0888102fd0 0x7f088819fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T19:02:53.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:53.442+0000 7f088fb81700 1 -- 172.21.15.32:0/3576033756 >> 172.21.15.32:0/3576033756 conn(0x7f0888079440 msgr2=0x7f0888101700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T19:02:53.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:53.442+0000 7f088fb81700 1 -- 172.21.15.32:0/3576033756 shutdown_connections 2024-05-03T19:02:53.445 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:53.442+0000 7f088fb81700 1 -- 172.21.15.32:0/3576033756 wait complete. 2024-05-03T19:02:53.445 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 26 2024-05-03T19:02:53.818 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:02:53 smithi032 ceph-mon[29763]: pgmap v825: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T19:02:53.819 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:02:53 smithi032 ceph-mon[29763]: from='client.? 172.21.15.32:0/3576033756' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T19:02:53.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 19:02:53 smithi045 ceph-mon[33299]: pgmap v825: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T19:02:53.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 19:02:53 smithi045 ceph-mon[33299]: from='client.? 172.21.15.32:0/3576033756' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T19:02:54.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 19:02:53 smithi113 ceph-mon[33540]: pgmap v825: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-03T19:02:54.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 19:02:53 smithi113 ceph-mon[33540]: from='client.? 172.21.15.32:0/3576033756' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T19:02:54.247 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid f500455e-097b-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-03T19:02:54.503 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T19:02:55.703 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:02:55 smithi032 ceph-mon[29763]: pgmap v826: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 341 B/s wr, 7 op/s 2024-05-03T19:02:55.853 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 19:02:55 smithi113 ceph-mon[33540]: pgmap v826: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 341 B/s wr, 7 op/s 2024-05-03T19:02:55.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 19:02:55 smithi045 ceph-mon[33299]: pgmap v826: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 341 B/s wr, 7 op/s 2024-05-03T19:02:56.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:56.209+0000 7f6c6f0f6700 1 -- 172.21.15.32:0/997617753 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c68108870 msgr2=0x7f6c681103f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T19:02:56.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:56.209+0000 7f6c6f0f6700 1 --2- 172.21.15.32:0/997617753 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c68108870 0x7f6c681103f0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f6c6000b0d0 tx=0x7f6c6000b3e0 comp rx=0 tx=0).stop 2024-05-03T19:02:56.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:56.209+0000 7f6c6f0f6700 1 -- 172.21.15.32:0/997617753 shutdown_connections 2024-05-03T19:02:56.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:56.209+0000 7f6c6f0f6700 1 --2- 172.21.15.32:0/997617753 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c68108870 0x7f6c681103f0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T19:02:56.211 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:56.209+0000 7f6c6f0f6700 1 --2- 172.21.15.32:0/997617753 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c68103780 0x7f6c68103b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T19:02:56.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:56.209+0000 7f6c6f0f6700 1 --2- 172.21.15.32:0/997617753 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6c68104130 0x7f6c68108140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T19:02:56.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:56.209+0000 7f6c6f0f6700 1 -- 172.21.15.32:0/997617753 >> 172.21.15.32:0/997617753 conn(0x7f6c6807a610 msgr2=0x7f6c6807aa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T19:02:56.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:56.209+0000 7f6c6f0f6700 1 -- 172.21.15.32:0/997617753 shutdown_connections 2024-05-03T19:02:56.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:56.209+0000 7f6c6f0f6700 1 -- 172.21.15.32:0/997617753 wait complete. 2024-05-03T19:02:56.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:56.209+0000 7f6c6f0f6700 1 Processor -- start 2024-05-03T19:02:56.212 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:56.210+0000 7f6c6f0f6700 1 -- start start 2024-05-03T19:02:56.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:56.210+0000 7f6c6f0f6700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c68103780 0x7f6c6819f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T19:02:56.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:56.210+0000 7f6c6f0f6700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6c68104130 0x7f6c6819fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T19:02:56.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:56.210+0000 7f6c6f0f6700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c68108870 0x7f6c681998e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T19:02:56.217 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:56.210+0000 7f6c6f0f6700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f6c68113660 con 0x7f6c68108870 2024-05-03T19:02:56.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:56.210+0000 7f6c6f0f6700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6c681134e0 con 0x7f6c68103780 2024-05-03T19:02:56.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:56.210+0000 7f6c6f0f6700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f6c681137e0 con 0x7f6c68104130 2024-05-03T19:02:56.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:56.210+0000 7f6c6d693700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c68108870 0x7f6c681998e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T19:02:56.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:56.210+0000 7f6c6d693700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c68108870 0x7f6c681998e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52752/0 (socket says 172.21.15.32:52752) 2024-05-03T19:02:56.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:56.210+0000 7f6c6d693700 1 -- 172.21.15.32:0/4091006750 learned_addr learned my addr 172.21.15.32:0/4091006750 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T19:02:56.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:56.210+0000 7f6c6ce92700 1 --2- 172.21.15.32:0/4091006750 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c68103780 0x7f6c6819f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T19:02:56.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:56.210+0000 7f6c67fff700 1 --2- 172.21.15.32:0/4091006750 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6c68104130 0x7f6c6819fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T19:02:56.218 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:56.210+0000 7f6c6d693700 1 -- 172.21.15.32:0/4091006750 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6c68104130 msgr2=0x7f6c6819fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T19:02:56.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:56.210+0000 7f6c6d693700 1 --2- 172.21.15.32:0/4091006750 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6c68104130 0x7f6c6819fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T19:02:56.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:56.210+0000 7f6c6d693700 1 -- 172.21.15.32:0/4091006750 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c68103780 msgr2=0x7f6c6819f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T19:02:56.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:56.210+0000 7f6c6d693700 1 --2- 172.21.15.32:0/4091006750 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c68103780 0x7f6c6819f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T19:02:56.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:56.210+0000 7f6c6d693700 1 -- 172.21.15.32:0/4091006750 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c6819a040 con 0x7f6c68108870 2024-05-03T19:02:56.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:56.210+0000 7f6c67fff700 1 --2- 172.21.15.32:0/4091006750 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6c68104130 0x7f6c6819fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T19:02:56.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:56.211+0000 7f6c6d693700 1 --2- 172.21.15.32:0/4091006750 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c68108870 0x7f6c681998e0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f6c600095c0 tx=0x7f6c60007ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T19:02:56.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:56.211+0000 7f6c65ffb700 1 -- 172.21.15.32:0/4091006750 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c6001f070 con 0x7f6c68108870 2024-05-03T19:02:56.219 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:56.211+0000 7f6c6f0f6700 1 -- 172.21.15.32:0/4091006750 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c6819a2d0 con 0x7f6c68108870 2024-05-03T19:02:56.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:56.211+0000 7f6c65ffb700 1 -- 172.21.15.32:0/4091006750 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c60003c00 con 0x7f6c68108870 2024-05-03T19:02:56.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:56.211+0000 7f6c65ffb700 1 -- 172.21.15.32:0/4091006750 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c6001b6a0 con 0x7f6c68108870 2024-05-03T19:02:56.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:56.211+0000 7f6c6f0f6700 1 -- 172.21.15.32:0/4091006750 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c681a65a0 con 0x7f6c68108870 2024-05-03T19:02:56.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:56.212+0000 7f6c6f0f6700 1 -- 172.21.15.32:0/4091006750 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c6806a190 con 0x7f6c68108870 2024-05-03T19:02:56.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:56.213+0000 7f6c65ffb700 1 -- 172.21.15.32:0/4091006750 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f6c60022070 con 0x7f6c68108870 2024-05-03T19:02:56.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:56.214+0000 7f6c65ffb700 1 --2- 172.21.15.32:0/4091006750 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f6c580b2990 0x7f6c580b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T19:02:56.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:56.214+0000 7f6c6ce92700 1 --2- 172.21.15.32:0/4091006750 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f6c580b2990 0x7f6c580b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T19:02:56.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:56.214+0000 7f6c65ffb700 1 -- 172.21.15.32:0/4091006750 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8356+0+0 (secure 0 0 0) 0x7f6c6000f020 con 0x7f6c68108870 2024-05-03T19:02:56.220 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:56.215+0000 7f6c6ce92700 1 --2- 172.21.15.32:0/4091006750 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f6c580b2990 0x7f6c580b4e50 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f6c68108370 tx=0x7f6c5c00b5d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T19:02:56.221 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:56.218+0000 7f6c65ffb700 1 -- 172.21.15.32:0/4091006750 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f6c60029020 con 0x7f6c68108870 2024-05-03T19:02:56.481 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:56.480+0000 7f6c6f0f6700 1 -- 172.21.15.32:0/4091006750 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6c6819adc0 con 0x7f6c68108870 2024-05-03T19:02:56.482 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:56.481+0000 7f6c65ffb700 1 -- 172.21.15.32:0/4091006750 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11862 (secure 0 0 0) 0x7f6c600ab710 con 0x7f6c68108870 2024-05-03T19:02:56.483 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-03T19:02:56.483 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/535599929","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":535599929},{"type":"v1","addr":"172.21.15.32:6835","nonce":535599929}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/3101679868","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":3101679868},{"type":"v1","addr":"172.21.15.32:6837","nonce":3101679868}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14610,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6839/3838653921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6838","nonce":3838653921},{"type":"v1","addr":"172.21.15.32:6839","nonce":3838653921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14622,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6841/3019648436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6840","nonce":3019648436},{"type":"v1","addr":"172.21.15.32:6841","nonce":3019648436}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24380,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/1629869435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":1629869435},{"type":"v1","addr":"172.21.15.45:6833","nonce":1629869435}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24398,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/2997291166","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":2997291166},{"type":"v1","addr":"172.21.15.45:6835","nonce":2997291166}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-03T18:44:51.155050+0000","modified":"2024-05-03T18:47:46.959629+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24478,"mds_1":24469,"mds_2":24463},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24425":{"gid":24425,"name":"h","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.45:6837/428055392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":428055392},{"type":"v1","addr":"172.21.15.45:6837","nonce":428055392}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.113:6833/3323558791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":3323558791},{"type":"v1","addr":"172.21.15.113:6833","nonce":3323558791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"b","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.45:6839/970804040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":970804040},{"type":"v1","addr":"172.21.15.45:6839","nonce":970804040}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.113:6835/2588738276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":2588738276},{"type":"v1","addr":"172.21.15.113:6835","nonce":2588738276}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24469":{"gid":24469,"name":"c","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.113:6837/3677284460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":3677284460},{"type":"v1","addr":"172.21.15.113:6837","nonce":3677284460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.113:6839/942050693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6838","nonce":942050693},{"type":"v1","addr":"172.21.15.113:6839","nonce":942050693}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24478,"qdb_cluster":[24463,24469,24478]},"id":1}]} 2024-05-03T19:02:56.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:56.483+0000 7f6c6f0f6700 1 -- 172.21.15.32:0/4091006750 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f6c580b2990 msgr2=0x7f6c580b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T19:02:56.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:56.483+0000 7f6c6f0f6700 1 --2- 172.21.15.32:0/4091006750 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f6c580b2990 0x7f6c580b4e50 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f6c68108370 tx=0x7f6c5c00b5d0 comp rx=0 tx=0).stop 2024-05-03T19:02:56.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:56.483+0000 7f6c6f0f6700 1 -- 172.21.15.32:0/4091006750 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c68108870 msgr2=0x7f6c681998e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T19:02:56.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:56.483+0000 7f6c6f0f6700 1 --2- 172.21.15.32:0/4091006750 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c68108870 0x7f6c681998e0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f6c600095c0 tx=0x7f6c60007ad0 comp rx=0 tx=0).stop 2024-05-03T19:02:56.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:56.483+0000 7f6c6f0f6700 1 -- 172.21.15.32:0/4091006750 shutdown_connections 2024-05-03T19:02:56.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:56.483+0000 7f6c6f0f6700 1 --2- 172.21.15.32:0/4091006750 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f6c580b2990 0x7f6c580b4e50 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T19:02:56.485 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:56.483+0000 7f6c6f0f6700 1 --2- 172.21.15.32:0/4091006750 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c68108870 0x7f6c681998e0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T19:02:56.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:56.483+0000 7f6c6f0f6700 1 --2- 172.21.15.32:0/4091006750 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c68103780 0x7f6c6819f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T19:02:56.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:56.483+0000 7f6c6f0f6700 1 --2- 172.21.15.32:0/4091006750 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6c68104130 0x7f6c6819fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T19:02:56.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:56.483+0000 7f6c6f0f6700 1 -- 172.21.15.32:0/4091006750 >> 172.21.15.32:0/4091006750 conn(0x7f6c6807a610 msgr2=0x7f6c681016d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T19:02:56.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:56.483+0000 7f6c6f0f6700 1 -- 172.21.15.32:0/4091006750 shutdown_connections 2024-05-03T19:02:56.486 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:56.483+0000 7f6c6f0f6700 1 -- 172.21.15.32:0/4091006750 wait complete. 2024-05-03T19:02:56.486 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 26 2024-05-03T19:02:56.809 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:02:56 smithi032 ceph-mon[29763]: from='client.? 172.21.15.32:0/4091006750' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T19:02:56.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 19:02:56 smithi045 ceph-mon[33299]: from='client.? 172.21.15.32:0/4091006750' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T19:02:57.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 19:02:56 smithi113 ceph-mon[33540]: from='client.? 172.21.15.32:0/4091006750' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-03T19:02:57.123 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid f500455e-097b-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-03T19:02:57.372 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T19:02:57.788 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:02:57 smithi032 ceph-mon[29763]: pgmap v827: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-05-03T19:02:57.899 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 19:02:57 smithi045 ceph-mon[33299]: pgmap v827: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-05-03T19:02:58.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 19:02:57 smithi113 ceph-mon[33540]: pgmap v827: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-05-03T19:02:58.901 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:58.899+0000 7f55a1826700 1 -- 172.21.15.32:0/486524673 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f559c104100 msgr2=0x7f559c108120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T19:02:58.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:58.899+0000 7f55a1826700 1 --2- 172.21.15.32:0/486524673 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f559c104100 0x7f559c108120 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f55900099c0 tx=0x7f5590009cd0 comp rx=0 tx=0).stop 2024-05-03T19:02:58.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:58.899+0000 7f55a1826700 1 -- 172.21.15.32:0/486524673 shutdown_connections 2024-05-03T19:02:58.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:58.899+0000 7f55a1826700 1 --2- 172.21.15.32:0/486524673 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f559c108850 0x7f559c1103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T19:02:58.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:58.899+0000 7f55a1826700 1 --2- 172.21.15.32:0/486524673 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f559c103750 0x7f559c103b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T19:02:58.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:58.899+0000 7f55a1826700 1 --2- 172.21.15.32:0/486524673 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f559c104100 0x7f559c108120 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T19:02:58.902 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:58.899+0000 7f55a1826700 1 -- 172.21.15.32:0/486524673 >> 172.21.15.32:0/486524673 conn(0x7f559c07a600 msgr2=0x7f559c07aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T19:02:58.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:58.899+0000 7f55a1826700 1 -- 172.21.15.32:0/486524673 shutdown_connections 2024-05-03T19:02:58.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:58.900+0000 7f55a1826700 1 -- 172.21.15.32:0/486524673 wait complete. 2024-05-03T19:02:58.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:58.900+0000 7f55a1826700 1 Processor -- start 2024-05-03T19:02:58.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:58.900+0000 7f55a1826700 1 -- start start 2024-05-03T19:02:58.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:58.900+0000 7f55a1826700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f559c103750 0x7f559c19f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T19:02:58.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:58.901+0000 7f55a1826700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f559c104100 0x7f559c19fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T19:02:58.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:58.901+0000 7f55a1826700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f559c108850 0x7f559c1998f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T19:02:58.903 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:58.901+0000 7f55a1826700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f559c113630 con 0x7f559c108850 2024-05-03T19:02:58.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:58.901+0000 7f55a1826700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f559c1134b0 con 0x7f559c103750 2024-05-03T19:02:58.904 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:58.901+0000 7f55a1826700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f559c1137b0 con 0x7f559c104100 2024-05-03T19:02:58.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:58.901+0000 7f559b7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f559c108850 0x7f559c1998f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T19:02:58.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:58.901+0000 7f559b7fe700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f559c108850 0x7f559c1998f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52780/0 (socket says 172.21.15.32:52780) 2024-05-03T19:02:58.905 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:58.901+0000 7f559b7fe700 1 -- 172.21.15.32:0/228984862 learned_addr learned my addr 172.21.15.32:0/228984862 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T19:02:58.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:58.901+0000 7f559affd700 1 --2- 172.21.15.32:0/228984862 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f559c103750 0x7f559c19f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T19:02:58.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:58.901+0000 7f559a7fc700 1 --2- 172.21.15.32:0/228984862 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f559c104100 0x7f559c19fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T19:02:58.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:58.901+0000 7f559b7fe700 1 -- 172.21.15.32:0/228984862 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f559c104100 msgr2=0x7f559c19fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T19:02:58.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:58.901+0000 7f559b7fe700 1 --2- 172.21.15.32:0/228984862 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f559c104100 0x7f559c19fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T19:02:58.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:58.901+0000 7f559b7fe700 1 -- 172.21.15.32:0/228984862 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f559c103750 msgr2=0x7f559c19f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T19:02:58.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:58.901+0000 7f559b7fe700 1 --2- 172.21.15.32:0/228984862 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f559c103750 0x7f559c19f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T19:02:58.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:58.901+0000 7f559b7fe700 1 -- 172.21.15.32:0/228984862 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f559c19a050 con 0x7f559c108850 2024-05-03T19:02:58.906 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:58.901+0000 7f559affd700 1 --2- 172.21.15.32:0/228984862 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f559c103750 0x7f559c19f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T19:02:58.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:58.901+0000 7f559b7fe700 1 --2- 172.21.15.32:0/228984862 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f559c108850 0x7f559c1998f0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f5594007ee0 tx=0x7f559400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T19:02:58.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:58.901+0000 7f55a0824700 1 -- 172.21.15.32:0/228984862 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f559400dbd0 con 0x7f559c108850 2024-05-03T19:02:58.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:58.901+0000 7f55a1826700 1 -- 172.21.15.32:0/228984862 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f559c19a340 con 0x7f559c108850 2024-05-03T19:02:58.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:58.901+0000 7f55a0824700 1 -- 172.21.15.32:0/228984862 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f559400dd70 con 0x7f559c108850 2024-05-03T19:02:58.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:58.901+0000 7f55a0824700 1 -- 172.21.15.32:0/228984862 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5594005990 con 0x7f559c108850 2024-05-03T19:02:58.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:58.902+0000 7f55a1826700 1 -- 172.21.15.32:0/228984862 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f559c1a6600 con 0x7f559c108850 2024-05-03T19:02:58.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:58.902+0000 7f55a1826700 1 -- 172.21.15.32:0/228984862 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f559c06a190 con 0x7f559c108850 2024-05-03T19:02:58.907 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:58.904+0000 7f55a0824700 1 -- 172.21.15.32:0/228984862 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f559401a020 con 0x7f559c108850 2024-05-03T19:02:58.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:58.905+0000 7f55a0824700 1 --2- 172.21.15.32:0/228984862 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f55840b2910 0x7f55840b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T19:02:58.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:58.905+0000 7f559affd700 1 --2- 172.21.15.32:0/228984862 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f55840b2910 0x7f55840b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T19:02:58.908 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:58.905+0000 7f55a0824700 1 -- 172.21.15.32:0/228984862 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8356+0+0 (secure 0 0 0) 0x7f55940e2390 con 0x7f559c108850 2024-05-03T19:02:58.909 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:58.905+0000 7f559affd700 1 --2- 172.21.15.32:0/228984862 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f55840b2910 0x7f55840b4dd0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f558c005950 tx=0x7f558c0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T19:02:58.910 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:58.908+0000 7f55a0824700 1 -- 172.21.15.32:0/228984862 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f5594010070 con 0x7f559c108850 2024-05-03T19:02:59.128 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:59.127+0000 7f55a1826700 1 -- 172.21.15.32:0/228984862 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f559c02e510 con 0x7f559c108850 2024-05-03T19:02:59.130 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:59.129+0000 7f55a0824700 1 -- 172.21.15.32:0/228984862 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+22814 (secure 0 0 0) 0x7f55940a9ed0 con 0x7f559c108850 2024-05-03T19:02:59.130 INFO:teuthology.orchestra.run.smithi032.stdout: 2024-05-03T19:02:59.131 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":82,"fsid":"f500455e-097b-11ef-bc97-c7b262605968","created":"2024-05-03T18:36:43.750772+0000","modified":"2024-05-03T18:46:47.787519+0000","last_up_change":"2024-05-03T18:43:50.933818+0000","last_in_change":"2024-05-03T18:43:31.804386+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-03T18:40:07.981853+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-03T18:44:44.607622+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-03T18:44:48.077391+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0430c3c0-2c50-471c-9c74-2869a0627c74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":262324956},{"type":"v1","addr":"172.21.15.32:6803","nonce":262324956}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":262324956},{"type":"v1","addr":"172.21.15.32:6805","nonce":262324956}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":262324956},{"type":"v1","addr":"172.21.15.32:6809","nonce":262324956}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":262324956},{"type":"v1","addr":"172.21.15.32:6807","nonce":262324956}]},"public_addr":"172.21.15.32:6803/262324956","cluster_addr":"172.21.15.32:6805/262324956","heartbeat_back_addr":"172.21.15.32:6809/262324956","heartbeat_front_addr":"172.21.15.32:6807/262324956","state":["exists","up"]},{"osd":1,"uuid":"ba8f0b67-1d7d-4d4c-8276-eff664d21f02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":1209907302},{"type":"v1","addr":"172.21.15.45:6801","nonce":1209907302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":1209907302},{"type":"v1","addr":"172.21.15.45:6803","nonce":1209907302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":1209907302},{"type":"v1","addr":"172.21.15.45:6807","nonce":1209907302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":1209907302},{"type":"v1","addr":"172.21.15.45:6805","nonce":1209907302}]},"public_addr":"172.21.15.45:6801/1209907302","cluster_addr":"172.21.15.45:6803/1209907302","heartbeat_back_addr":"172.21.15.45:6807/1209907302","heartbeat_front_addr":"172.21.15.45:6805/1209907302","state":["exists","up"]},{"osd":2,"uuid":"5624eabc-be47-4bb8-9d8e-b84734769c83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":2308128926},{"type":"v1","addr":"172.21.15.113:6801","nonce":2308128926}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":2308128926},{"type":"v1","addr":"172.21.15.113:6803","nonce":2308128926}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":2308128926},{"type":"v1","addr":"172.21.15.113:6807","nonce":2308128926}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":2308128926},{"type":"v1","addr":"172.21.15.113:6805","nonce":2308128926}]},"public_addr":"172.21.15.113:6801/2308128926","cluster_addr":"172.21.15.113:6803/2308128926","heartbeat_back_addr":"172.21.15.113:6807/2308128926","heartbeat_front_addr":"172.21.15.113:6805/2308128926","state":["exists","up"]},{"osd":3,"uuid":"62c9c977-afe1-4112-beba-b25af7caa2f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":2606514063},{"type":"v1","addr":"172.21.15.32:6811","nonce":2606514063}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":2606514063},{"type":"v1","addr":"172.21.15.32:6813","nonce":2606514063}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":2606514063},{"type":"v1","addr":"172.21.15.32:6817","nonce":2606514063}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":2606514063},{"type":"v1","addr":"172.21.15.32:6815","nonce":2606514063}]},"public_addr":"172.21.15.32:6811/2606514063","cluster_addr":"172.21.15.32:6813/2606514063","heartbeat_back_addr":"172.21.15.32:6817/2606514063","heartbeat_front_addr":"172.21.15.32:6815/2606514063","state":["exists","up"]},{"osd":4,"uuid":"99ad449f-b0bc-4aa1-b0cf-4a7a458126e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":71932118},{"type":"v1","addr":"172.21.15.45:6809","nonce":71932118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":71932118},{"type":"v1","addr":"172.21.15.45:6811","nonce":71932118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":71932118},{"type":"v1","addr":"172.21.15.45:6815","nonce":71932118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":71932118},{"type":"v1","addr":"172.21.15.45:6813","nonce":71932118}]},"public_addr":"172.21.15.45:6809/71932118","cluster_addr":"172.21.15.45:6811/71932118","heartbeat_back_addr":"172.21.15.45:6815/71932118","heartbeat_front_addr":"172.21.15.45:6813/71932118","state":["exists","up"]},{"osd":5,"uuid":"a6688ade-47d7-4a52-807f-3c880a8ffcbb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":4085018597},{"type":"v1","addr":"172.21.15.113:6809","nonce":4085018597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":4085018597},{"type":"v1","addr":"172.21.15.113:6811","nonce":4085018597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":4085018597},{"type":"v1","addr":"172.21.15.113:6815","nonce":4085018597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":4085018597},{"type":"v1","addr":"172.21.15.113:6813","nonce":4085018597}]},"public_addr":"172.21.15.113:6809/4085018597","cluster_addr":"172.21.15.113:6811/4085018597","heartbeat_back_addr":"172.21.15.113:6815/4085018597","heartbeat_front_addr":"172.21.15.113:6813/4085018597","state":["exists","up"]},{"osd":6,"uuid":"1d9ebda9-55c7-473a-9dc0-fc60845bbfe9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":791407796},{"type":"v1","addr":"172.21.15.32:6819","nonce":791407796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6820","nonce":791407796},{"type":"v1","addr":"172.21.15.32:6821","nonce":791407796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6824","nonce":791407796},{"type":"v1","addr":"172.21.15.32:6825","nonce":791407796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":791407796},{"type":"v1","addr":"172.21.15.32:6823","nonce":791407796}]},"public_addr":"172.21.15.32:6819/791407796","cluster_addr":"172.21.15.32:6821/791407796","heartbeat_back_addr":"172.21.15.32:6825/791407796","heartbeat_front_addr":"172.21.15.32:6823/791407796","state":["exists","up"]},{"osd":7,"uuid":"988cce65-23ed-4814-ac09-177eb2f89fe1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1032572438},{"type":"v1","addr":"172.21.15.45:6817","nonce":1032572438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1032572438},{"type":"v1","addr":"172.21.15.45:6819","nonce":1032572438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1032572438},{"type":"v1","addr":"172.21.15.45:6823","nonce":1032572438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1032572438},{"type":"v1","addr":"172.21.15.45:6821","nonce":1032572438}]},"public_addr":"172.21.15.45:6817/1032572438","cluster_addr":"172.21.15.45:6819/1032572438","heartbeat_back_addr":"172.21.15.45:6823/1032572438","heartbeat_front_addr":"172.21.15.45:6821/1032572438","state":["exists","up"]},{"osd":8,"uuid":"816e8ce8-5afa-4b52-89be-f7793ed12a0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":3569181558},{"type":"v1","addr":"172.21.15.113:6817","nonce":3569181558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":3569181558},{"type":"v1","addr":"172.21.15.113:6819","nonce":3569181558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":3569181558},{"type":"v1","addr":"172.21.15.113:6823","nonce":3569181558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":3569181558},{"type":"v1","addr":"172.21.15.113:6821","nonce":3569181558}]},"public_addr":"172.21.15.113:6817/3569181558","cluster_addr":"172.21.15.113:6819/3569181558","heartbeat_back_addr":"172.21.15.113:6823/3569181558","heartbeat_front_addr":"172.21.15.113:6821/3569181558","state":["exists","up"]},{"osd":9,"uuid":"1d35c435-36b8-40e8-a2d4-0efe904f1107","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":4290196345},{"type":"v1","addr":"172.21.15.32:6827","nonce":4290196345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6828","nonce":4290196345},{"type":"v1","addr":"172.21.15.32:6829","nonce":4290196345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":4290196345},{"type":"v1","addr":"172.21.15.32:6833","nonce":4290196345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":4290196345},{"type":"v1","addr":"172.21.15.32:6831","nonce":4290196345}]},"public_addr":"172.21.15.32:6827/4290196345","cluster_addr":"172.21.15.32:6829/4290196345","heartbeat_back_addr":"172.21.15.32:6833/4290196345","heartbeat_front_addr":"172.21.15.32:6831/4290196345","state":["exists","up"]},{"osd":10,"uuid":"9af7879e-aa8d-4ef8-b289-4b145b614820","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":1817064361},{"type":"v1","addr":"172.21.15.45:6825","nonce":1817064361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":1817064361},{"type":"v1","addr":"172.21.15.45:6827","nonce":1817064361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":1817064361},{"type":"v1","addr":"172.21.15.45:6831","nonce":1817064361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":1817064361},{"type":"v1","addr":"172.21.15.45:6829","nonce":1817064361}]},"public_addr":"172.21.15.45:6825/1817064361","cluster_addr":"172.21.15.45:6827/1817064361","heartbeat_back_addr":"172.21.15.45:6831/1817064361","heartbeat_front_addr":"172.21.15.45:6829/1817064361","state":["exists","up"]},{"osd":11,"uuid":"22981709-1f9d-46f6-892a-5678394a2e1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":703464306},{"type":"v1","addr":"172.21.15.113:6825","nonce":703464306}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":703464306},{"type":"v1","addr":"172.21.15.113:6827","nonce":703464306}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":703464306},{"type":"v1","addr":"172.21.15.113:6831","nonce":703464306}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":703464306},{"type":"v1","addr":"172.21.15.113:6829","nonce":703464306}]},"public_addr":"172.21.15.113:6825/703464306","cluster_addr":"172.21.15.113:6827/703464306","heartbeat_back_addr":"172.21.15.113:6831/703464306","heartbeat_front_addr":"172.21.15.113:6829/703464306","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:39:18.455575+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:39:42.234466+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:40:05.843072+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:40:30.822533+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:40:54.952487+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:41:18.750184+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:41:43.403326+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:42:08.285730+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:42:32.861740+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:42:59.307277+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:43:24.509691+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-03T18:43:49.850077+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":2,"to":1}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.c","mappings":[{"from":5,"to":8}]},{"pgid":"3.e","mappings":[{"from":4,"to":7}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.1b","mappings":[{"from":4,"to":6}]},{"pgid":"3.2f","mappings":[{"from":1,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"3.3b","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:0/743758274":"2024-05-04T18:37:45.879602+0000","172.21.15.32:6801/2019702098":"2024-05-04T18:37:45.879602+0000","172.21.15.32:6800/2019702098":"2024-05-04T18:37:45.879602+0000","172.21.15.32:0/3192385439":"2024-05-04T18:37:18.492319+0000","172.21.15.32:0/1450308494":"2024-05-04T18:37:18.492319+0000","172.21.15.32:0/3957699240":"2024-05-04T18:37:45.879602+0000","172.21.15.32:0/3310503393":"2024-05-04T18:37:18.492319+0000","172.21.15.32:6801/3945405684":"2024-05-04T18:37:18.492319+0000","172.21.15.32:6800/3945405684":"2024-05-04T18:37:18.492319+0000","172.21.15.32:0/2471164468":"2024-05-04T18:37:45.879602+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-03T19:02:59.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:59.131+0000 7f55a1826700 1 -- 172.21.15.32:0/228984862 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f55840b2910 msgr2=0x7f55840b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T19:02:59.132 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:59.131+0000 7f55a1826700 1 --2- 172.21.15.32:0/228984862 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f55840b2910 0x7f55840b4dd0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f558c005950 tx=0x7f558c0058e0 comp rx=0 tx=0).stop 2024-05-03T19:02:59.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:59.131+0000 7f55a1826700 1 -- 172.21.15.32:0/228984862 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f559c108850 msgr2=0x7f559c1998f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T19:02:59.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:59.131+0000 7f55a1826700 1 --2- 172.21.15.32:0/228984862 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f559c108850 0x7f559c1998f0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f5594007ee0 tx=0x7f559400d3b0 comp rx=0 tx=0).stop 2024-05-03T19:02:59.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:59.131+0000 7f55a1826700 1 -- 172.21.15.32:0/228984862 shutdown_connections 2024-05-03T19:02:59.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:59.131+0000 7f55a1826700 1 --2- 172.21.15.32:0/228984862 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f55840b2910 0x7f55840b4dd0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T19:02:59.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:59.131+0000 7f55a1826700 1 --2- 172.21.15.32:0/228984862 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f559c108850 0x7f559c1998f0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T19:02:59.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:59.131+0000 7f55a1826700 1 --2- 172.21.15.32:0/228984862 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f559c103750 0x7f559c19f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T19:02:59.133 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:59.131+0000 7f55a1826700 1 --2- 172.21.15.32:0/228984862 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f559c104100 0x7f559c19fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T19:02:59.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:59.131+0000 7f55a1826700 1 -- 172.21.15.32:0/228984862 >> 172.21.15.32:0/228984862 conn(0x7f559c07a600 msgr2=0x7f559c1099a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T19:02:59.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:59.131+0000 7f55a1826700 1 -- 172.21.15.32:0/228984862 shutdown_connections 2024-05-03T19:02:59.134 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:02:59.131+0000 7f55a1826700 1 -- 172.21.15.32:0/228984862 wait complete. 2024-05-03T19:02:59.762 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:02:59 smithi032 ceph-mon[29763]: pgmap v828: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2024-05-03T19:02:59.762 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:02:59 smithi032 ceph-mon[29763]: from='client.? 172.21.15.32:0/228984862' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T19:02:59.764 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid f500455e-097b-11ef-bc97-c7b262605968 -- ceph fs fail cephfs 2024-05-03T19:02:59.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 19:02:59 smithi045 ceph-mon[33299]: pgmap v828: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2024-05-03T19:02:59.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 19:02:59 smithi045 ceph-mon[33299]: from='client.? 172.21.15.32:0/228984862' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T19:03:00.012 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T19:03:00.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 19:02:59 smithi113 ceph-mon[33540]: pgmap v828: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2024-05-03T19:03:00.052 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 19:02:59 smithi113 ceph-mon[33540]: from='client.? 172.21.15.32:0/228984862' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-03T19:03:01.779 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:03:01 smithi032 ceph-mon[29763]: pgmap v829: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-05-03T19:03:01.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:01.800+0000 7f9b2a94b700 1 -- 172.21.15.32:0/3243282189 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b24108870 msgr2=0x7f9b241103f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T19:03:01.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:01.800+0000 7f9b2a94b700 1 --2- 172.21.15.32:0/3243282189 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b24108870 0x7f9b241103f0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f9b1c00b0d0 tx=0x7f9b1c00b3e0 comp rx=0 tx=0).stop 2024-05-03T19:03:01.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:01.800+0000 7f9b2a94b700 1 -- 172.21.15.32:0/3243282189 shutdown_connections 2024-05-03T19:03:01.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:01.800+0000 7f9b2a94b700 1 --2- 172.21.15.32:0/3243282189 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b24108870 0x7f9b241103f0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T19:03:01.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:01.800+0000 7f9b2a94b700 1 --2- 172.21.15.32:0/3243282189 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9b24103780 0x7f9b24103b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T19:03:01.802 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:01.800+0000 7f9b2a94b700 1 --2- 172.21.15.32:0/3243282189 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9b24104130 0x7f9b24108140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T19:03:01.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:01.800+0000 7f9b2a94b700 1 -- 172.21.15.32:0/3243282189 >> 172.21.15.32:0/3243282189 conn(0x7f9b2407a610 msgr2=0x7f9b2407aa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T19:03:01.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:01.800+0000 7f9b2a94b700 1 -- 172.21.15.32:0/3243282189 shutdown_connections 2024-05-03T19:03:01.803 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:01.800+0000 7f9b2a94b700 1 -- 172.21.15.32:0/3243282189 wait complete. 2024-05-03T19:03:01.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:01.801+0000 7f9b2a94b700 1 Processor -- start 2024-05-03T19:03:01.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:01.801+0000 7f9b2a94b700 1 -- start start 2024-05-03T19:03:01.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:01.801+0000 7f9b2a94b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9b24103780 0x7f9b240757a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T19:03:01.805 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:01.801+0000 7f9b2a94b700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b24104130 0x7f9b24075ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T19:03:01.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:01.801+0000 7f9b2a94b700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9b24108870 0x7f9b24076220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T19:03:01.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:01.801+0000 7f9b2a94b700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f9b241136c0 con 0x7f9b24104130 2024-05-03T19:03:01.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:01.801+0000 7f9b2a94b700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f9b24113540 con 0x7f9b24103780 2024-05-03T19:03:01.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:01.801+0000 7f9b2a94b700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f9b24113840 con 0x7f9b24108870 2024-05-03T19:03:01.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:01.801+0000 7f9b1bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b24104130 0x7f9b24075ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T19:03:01.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:01.801+0000 7f9b1bfff700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b24104130 0x7f9b24075ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52790/0 (socket says 172.21.15.32:52790) 2024-05-03T19:03:01.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:01.801+0000 7f9b1bfff700 1 -- 172.21.15.32:0/1740069805 learned_addr learned my addr 172.21.15.32:0/1740069805 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T19:03:01.806 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:01.801+0000 7f9b23fff700 1 --2- 172.21.15.32:0/1740069805 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9b24103780 0x7f9b240757a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T19:03:01.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:01.802+0000 7f9b28ee8700 1 --2- 172.21.15.32:0/1740069805 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9b24108870 0x7f9b24076220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T19:03:01.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:01.802+0000 7f9b1bfff700 1 -- 172.21.15.32:0/1740069805 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9b24108870 msgr2=0x7f9b24076220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T19:03:01.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:01.802+0000 7f9b1bfff700 1 --2- 172.21.15.32:0/1740069805 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9b24108870 0x7f9b24076220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T19:03:01.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:01.802+0000 7f9b1bfff700 1 -- 172.21.15.32:0/1740069805 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9b24103780 msgr2=0x7f9b240757a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T19:03:01.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:01.802+0000 7f9b1bfff700 1 --2- 172.21.15.32:0/1740069805 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9b24103780 0x7f9b240757a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T19:03:01.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:01.802+0000 7f9b1bfff700 1 -- 172.21.15.32:0/1740069805 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b241a8020 con 0x7f9b24104130 2024-05-03T19:03:01.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:01.802+0000 7f9b23fff700 1 --2- 172.21.15.32:0/1740069805 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9b24103780 0x7f9b240757a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-03T19:03:01.807 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:01.802+0000 7f9b28ee8700 1 --2- 172.21.15.32:0/1740069805 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9b24108870 0x7f9b24076220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T19:03:01.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:01.802+0000 7f9b1bfff700 1 --2- 172.21.15.32:0/1740069805 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b24104130 0x7f9b24075ce0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f9b0c00d980 tx=0x7f9b0c00dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T19:03:01.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:01.802+0000 7f9b21ffb700 1 -- 172.21.15.32:0/1740069805 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b0c016070 con 0x7f9b24104130 2024-05-03T19:03:01.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:01.802+0000 7f9b21ffb700 1 -- 172.21.15.32:0/1740069805 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9b0c0044d0 con 0x7f9b24104130 2024-05-03T19:03:01.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:01.802+0000 7f9b2a94b700 1 -- 172.21.15.32:0/1740069805 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b241a8310 con 0x7f9b24104130 2024-05-03T19:03:01.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:01.802+0000 7f9b21ffb700 1 -- 172.21.15.32:0/1740069805 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b0c004d00 con 0x7f9b24104130 2024-05-03T19:03:01.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:01.802+0000 7f9b2a94b700 1 -- 172.21.15.32:0/1740069805 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b241a8850 con 0x7f9b24104130 2024-05-03T19:03:01.808 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:01.803+0000 7f9b2a94b700 1 -- 172.21.15.32:0/1740069805 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b2406a190 con 0x7f9b24104130 2024-05-03T19:03:01.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:01.805+0000 7f9b21ffb700 1 -- 172.21.15.32:0/1740069805 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f9b0c004020 con 0x7f9b24104130 2024-05-03T19:03:01.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:01.806+0000 7f9b21ffb700 1 --2- 172.21.15.32:0/1740069805 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f9b100b2910 0x7f9b100b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T19:03:01.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:01.806+0000 7f9b23fff700 1 --2- 172.21.15.32:0/1740069805 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f9b100b2910 0x7f9b100b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T19:03:01.809 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:01.806+0000 7f9b21ffb700 1 -- 172.21.15.32:0/1740069805 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8356+0+0 (secure 0 0 0) 0x7f9b0c0e28c0 con 0x7f9b24104130 2024-05-03T19:03:01.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:01.806+0000 7f9b23fff700 1 --2- 172.21.15.32:0/1740069805 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f9b100b2910 0x7f9b100b4dd0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f9b140099d0 tx=0x7f9b14008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T19:03:01.810 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:01.809+0000 7f9b21ffb700 1 -- 172.21.15.32:0/1740069805 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f9b0c01f020 con 0x7f9b24104130 2024-05-03T19:03:01.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 19:03:01 smithi045 ceph-mon[33299]: pgmap v829: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-05-03T19:03:02.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 19:03:01 smithi113 ceph-mon[33540]: pgmap v829: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-05-03T19:03:02.072 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:02.070+0000 7f9b2a94b700 1 -- 172.21.15.32:0/1740069805 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f9b2410bab0 con 0x7f9b24104130 2024-05-03T19:03:02.074 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:02.073+0000 7f9b21ffb700 1 -- 172.21.15.32:0/1740069805 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v27) ==== 161+0+0 (secure 0 0 0) 0x7f9b0c055e20 con 0x7f9b24104130 2024-05-03T19:03:02.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:02.076+0000 7f9b2a94b700 1 -- 172.21.15.32:0/1740069805 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f9b100b2910 msgr2=0x7f9b100b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T19:03:02.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:02.076+0000 7f9b2a94b700 1 --2- 172.21.15.32:0/1740069805 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f9b100b2910 0x7f9b100b4dd0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f9b140099d0 tx=0x7f9b14008040 comp rx=0 tx=0).stop 2024-05-03T19:03:02.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:02.076+0000 7f9b2a94b700 1 -- 172.21.15.32:0/1740069805 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b24104130 msgr2=0x7f9b24075ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T19:03:02.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:02.076+0000 7f9b2a94b700 1 --2- 172.21.15.32:0/1740069805 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b24104130 0x7f9b24075ce0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f9b0c00d980 tx=0x7f9b0c00dd40 comp rx=0 tx=0).stop 2024-05-03T19:03:02.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:02.076+0000 7f9b2a94b700 1 -- 172.21.15.32:0/1740069805 shutdown_connections 2024-05-03T19:03:02.078 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:02.076+0000 7f9b2a94b700 1 --2- 172.21.15.32:0/1740069805 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f9b100b2910 0x7f9b100b4dd0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T19:03:02.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:02.076+0000 7f9b2a94b700 1 --2- 172.21.15.32:0/1740069805 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9b24108870 0x7f9b24076220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T19:03:02.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:02.076+0000 7f9b2a94b700 1 --2- 172.21.15.32:0/1740069805 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9b24103780 0x7f9b240757a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T19:03:02.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:02.076+0000 7f9b2a94b700 1 --2- 172.21.15.32:0/1740069805 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b24104130 0x7f9b24075ce0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T19:03:02.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:02.076+0000 7f9b2a94b700 1 -- 172.21.15.32:0/1740069805 >> 172.21.15.32:0/1740069805 conn(0x7f9b2407a610 msgr2=0x7f9b24102ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T19:03:02.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:02.076+0000 7f9b2a94b700 1 -- 172.21.15.32:0/1740069805 shutdown_connections 2024-05-03T19:03:02.079 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:02.076+0000 7f9b2a94b700 1 -- 172.21.15.32:0/1740069805 wait complete. 2024-05-03T19:03:02.080 INFO:teuthology.orchestra.run.smithi032.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-03T19:03:02.374 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:03:02 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-mon-a[29759]: 2024-05-03T19:03:02.071+0000 7f0508243700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-03T19:03:02.498 INFO:journalctl@ceph.mds.b.smithi045.stdout:May 03 19:03:02 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-mds-b[63790]: ignoring --setuser ceph since I am not root 2024-05-03T19:03:02.498 INFO:journalctl@ceph.mds.b.smithi045.stdout:May 03 19:03:02 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-mds-b[63790]: ignoring --setgroup ceph since I am not root 2024-05-03T19:03:02.499 INFO:journalctl@ceph.mds.b.smithi045.stdout:May 03 19:03:02 smithi045 ceph-mds[63794]: -- 172.21.15.45:0/682286857 <== mon.0 v2:172.21.15.32:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55fb8f419a40 con 0x55fb8f3ed000 2024-05-03T19:03:02.499 INFO:journalctl@ceph.mds.b.smithi045.stdout:May 03 19:03:02 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-mds-b[63790]: starting mds.b at 2024-05-03T19:03:02.499 INFO:journalctl@ceph.mds.h.smithi045.stdout:May 03 19:03:02 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-mds-h[63473]: ignoring --setuser ceph since I am not root 2024-05-03T19:03:02.499 INFO:journalctl@ceph.mds.h.smithi045.stdout:May 03 19:03:02 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-mds-h[63473]: ignoring --setgroup ceph since I am not root 2024-05-03T19:03:02.499 INFO:journalctl@ceph.mds.h.smithi045.stdout:May 03 19:03:02 smithi045 ceph-mds[63477]: -- 172.21.15.45:0/273446248 <== mon.1 v2:172.21.15.113:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5605e1091a40 con 0x5605e1064800 2024-05-03T19:03:02.500 INFO:journalctl@ceph.mds.h.smithi045.stdout:May 03 19:03:02 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-mds-h[63473]: starting mds.h at 2024-05-03T19:03:02.500 INFO:journalctl@ceph.mds.h.smithi045.stdout:May 03 19:03:02 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-mds-h[63473]: 2024-05-03T19:03:02.121+0000 7fa1a4015700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6836/0: (98) Address already in use 2024-05-03T19:03:02.552 INFO:journalctl@ceph.mds.l.smithi113.stdout:May 03 19:03:02 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-mds-l[64287]: ignoring --setuser ceph since I am not root 2024-05-03T19:03:02.552 INFO:journalctl@ceph.mds.l.smithi113.stdout:May 03 19:03:02 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-mds-l[64287]: ignoring --setgroup ceph since I am not root 2024-05-03T19:03:02.552 INFO:journalctl@ceph.mds.l.smithi113.stdout:May 03 19:03:02 smithi113 ceph-mds[64291]: -- 172.21.15.113:0/4243717580 <== mon.1 v2:172.21.15.113:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563a68b11a40 con 0x563a68ae4800 2024-05-03T19:03:02.552 INFO:journalctl@ceph.mds.l.smithi113.stdout:May 03 19:03:02 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-mds-l[64287]: starting mds.l at 2024-05-03T19:03:02.553 INFO:journalctl@ceph.mds.f.smithi113.stdout:May 03 19:03:02 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-mds-f[64599]: ignoring --setuser ceph since I am not root 2024-05-03T19:03:02.553 INFO:journalctl@ceph.mds.f.smithi113.stdout:May 03 19:03:02 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-mds-f[64599]: ignoring --setgroup ceph since I am not root 2024-05-03T19:03:02.553 INFO:journalctl@ceph.mds.f.smithi113.stdout:May 03 19:03:02 smithi113 ceph-mds[64603]: -- 172.21.15.113:0/4100719636 <== mon.1 v2:172.21.15.113:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c6e518ba40 con 0x55c6e515ec00 2024-05-03T19:03:02.553 INFO:journalctl@ceph.mds.f.smithi113.stdout:May 03 19:03:02 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-mds-f[64599]: starting mds.f at 2024-05-03T19:03:02.553 INFO:journalctl@ceph.mds.c.smithi113.stdout:May 03 19:03:02 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-mds-c[64907]: ignoring --setuser ceph since I am not root 2024-05-03T19:03:02.554 INFO:journalctl@ceph.mds.c.smithi113.stdout:May 03 19:03:02 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-mds-c[64907]: ignoring --setgroup ceph since I am not root 2024-05-03T19:03:02.554 INFO:journalctl@ceph.mds.c.smithi113.stdout:May 03 19:03:02 smithi113 ceph-mds[64911]: -- 172.21.15.113:0/2632271262 <== mon.1 v2:172.21.15.113:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558a3cc07a40 con 0x558a3cbda800 2024-05-03T19:03:02.554 INFO:journalctl@ceph.mds.c.smithi113.stdout:May 03 19:03:02 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-mds-c[64907]: starting mds.c at 2024-05-03T19:03:02.554 INFO:journalctl@ceph.mds.i.smithi113.stdout:May 03 19:03:02 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-mds-i[65223]: ignoring --setuser ceph since I am not root 2024-05-03T19:03:02.554 INFO:journalctl@ceph.mds.i.smithi113.stdout:May 03 19:03:02 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-mds-i[65223]: ignoring --setgroup ceph since I am not root 2024-05-03T19:03:02.554 INFO:journalctl@ceph.mds.i.smithi113.stdout:May 03 19:03:02 smithi113 ceph-mds[65227]: -- 172.21.15.113:0/2127722789 <== mon.1 v2:172.21.15.113:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5605a5283a40 con 0x5605a5256800 2024-05-03T19:03:02.554 INFO:journalctl@ceph.mds.i.smithi113.stdout:May 03 19:03:02 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-mds-i[65223]: starting mds.i at 2024-05-03T19:03:02.666 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:03:02 smithi032 ceph-mon[29763]: from='client.? 172.21.15.32:0/1740069805' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-03T19:03:02.667 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:03:02 smithi032 ceph-mon[29763]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-03T19:03:02.667 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:03:02 smithi032 ceph-mon[29763]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-03T19:03:02.667 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:03:02 smithi032 ceph-mon[29763]: from='client.? 172.21.15.32:0/1740069805' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-03T19:03:02.667 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:03:02 smithi032 ceph-mon[29763]: osdmap e83: 12 total, 12 up, 12 in 2024-05-03T19:03:02.667 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:03:02 smithi032 ceph-mon[29763]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-05-03T19:03:02.697 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid f500455e-097b-11ef-bc97-c7b262605968 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-05-03T19:03:02.944 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T19:03:02.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 19:03:02 smithi045 ceph-mon[33299]: from='client.? 172.21.15.32:0/1740069805' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-03T19:03:02.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 19:03:02 smithi045 ceph-mon[33299]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-03T19:03:02.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 19:03:02 smithi045 ceph-mon[33299]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-03T19:03:02.999 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 19:03:02 smithi045 ceph-mon[33299]: from='client.? 172.21.15.32:0/1740069805' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-03T19:03:02.999 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 19:03:02 smithi045 ceph-mon[33299]: osdmap e83: 12 total, 12 up, 12 in 2024-05-03T19:03:02.999 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 19:03:02 smithi045 ceph-mon[33299]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-05-03T19:03:03.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 19:03:02 smithi113 ceph-mon[33540]: from='client.? 172.21.15.32:0/1740069805' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-03T19:03:03.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 19:03:02 smithi113 ceph-mon[33540]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-03T19:03:03.052 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 19:03:02 smithi113 ceph-mon[33540]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-03T19:03:03.052 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 19:03:02 smithi113 ceph-mon[33540]: from='client.? 172.21.15.32:0/1740069805' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-03T19:03:03.052 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 19:03:02 smithi113 ceph-mon[33540]: osdmap e83: 12 total, 12 up, 12 in 2024-05-03T19:03:03.052 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 19:03:02 smithi113 ceph-mon[33540]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-05-03T19:03:03.819 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:03:03 smithi032 ceph-mon[29763]: pgmap v831: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 511 B/s wr, 6 op/s 2024-05-03T19:03:03.819 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:03:03 smithi032 ceph-mon[29763]: mds.? [v2:172.21.15.113:6832/14059280,v1:172.21.15.113:6834/14059280] up:boot 2024-05-03T19:03:03.819 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:03:03 smithi032 ceph-mon[29763]: mds.? [v2:172.21.15.113:6833/607603173,v1:172.21.15.113:6835/607603173] up:boot 2024-05-03T19:03:03.819 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:03:03 smithi032 ceph-mon[29763]: mds.? [v2:172.21.15.45:6836/3559059537,v1:172.21.15.45:6838/3559059537] up:boot 2024-05-03T19:03:03.819 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:03:03 smithi032 ceph-mon[29763]: mds.? [v2:172.21.15.45:6837/2434874026,v1:172.21.15.45:6839/2434874026] up:boot 2024-05-03T19:03:03.819 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:03:03 smithi032 ceph-mon[29763]: mds.? [v2:172.21.15.113:6836/591104415,v1:172.21.15.113:6838/591104415] up:boot 2024-05-03T19:03:03.820 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:03:03 smithi032 ceph-mon[29763]: mds.? [v2:172.21.15.113:6837/1635138625,v1:172.21.15.113:6839/1635138625] up:boot 2024-05-03T19:03:03.820 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:03:03 smithi032 ceph-mon[29763]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-05-03T19:03:03.820 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:03:03 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-03T19:03:03.820 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:03:03 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-03T19:03:03.820 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:03:03 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-03T19:03:03.820 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:03:03 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-03T19:03:03.820 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:03:03 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-03T19:03:03.820 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:03:03 smithi032 ceph-mon[29763]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-03T19:03:03.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 19:03:03 smithi045 ceph-mon[33299]: pgmap v831: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 511 B/s wr, 6 op/s 2024-05-03T19:03:03.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 19:03:03 smithi045 ceph-mon[33299]: mds.? [v2:172.21.15.113:6832/14059280,v1:172.21.15.113:6834/14059280] up:boot 2024-05-03T19:03:03.999 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 19:03:03 smithi045 ceph-mon[33299]: mds.? [v2:172.21.15.113:6833/607603173,v1:172.21.15.113:6835/607603173] up:boot 2024-05-03T19:03:03.999 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 19:03:03 smithi045 ceph-mon[33299]: mds.? [v2:172.21.15.45:6836/3559059537,v1:172.21.15.45:6838/3559059537] up:boot 2024-05-03T19:03:03.999 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 19:03:03 smithi045 ceph-mon[33299]: mds.? [v2:172.21.15.45:6837/2434874026,v1:172.21.15.45:6839/2434874026] up:boot 2024-05-03T19:03:03.999 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 19:03:03 smithi045 ceph-mon[33299]: mds.? [v2:172.21.15.113:6836/591104415,v1:172.21.15.113:6838/591104415] up:boot 2024-05-03T19:03:03.999 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 19:03:03 smithi045 ceph-mon[33299]: mds.? [v2:172.21.15.113:6837/1635138625,v1:172.21.15.113:6839/1635138625] up:boot 2024-05-03T19:03:03.999 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 19:03:03 smithi045 ceph-mon[33299]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-05-03T19:03:03.999 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 19:03:03 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-03T19:03:03.999 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 19:03:03 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-03T19:03:04.000 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 19:03:03 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-03T19:03:04.000 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 19:03:03 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-03T19:03:04.000 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 19:03:03 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-03T19:03:04.000 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 19:03:03 smithi045 ceph-mon[33299]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-03T19:03:04.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 19:03:03 smithi113 ceph-mon[33540]: pgmap v831: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 511 B/s wr, 6 op/s 2024-05-03T19:03:04.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 19:03:03 smithi113 ceph-mon[33540]: mds.? [v2:172.21.15.113:6832/14059280,v1:172.21.15.113:6834/14059280] up:boot 2024-05-03T19:03:04.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 19:03:03 smithi113 ceph-mon[33540]: mds.? [v2:172.21.15.113:6833/607603173,v1:172.21.15.113:6835/607603173] up:boot 2024-05-03T19:03:04.052 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 19:03:03 smithi113 ceph-mon[33540]: mds.? [v2:172.21.15.45:6836/3559059537,v1:172.21.15.45:6838/3559059537] up:boot 2024-05-03T19:03:04.052 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 19:03:03 smithi113 ceph-mon[33540]: mds.? [v2:172.21.15.45:6837/2434874026,v1:172.21.15.45:6839/2434874026] up:boot 2024-05-03T19:03:04.052 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 19:03:03 smithi113 ceph-mon[33540]: mds.? [v2:172.21.15.113:6836/591104415,v1:172.21.15.113:6838/591104415] up:boot 2024-05-03T19:03:04.052 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 19:03:03 smithi113 ceph-mon[33540]: mds.? [v2:172.21.15.113:6837/1635138625,v1:172.21.15.113:6839/1635138625] up:boot 2024-05-03T19:03:04.052 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 19:03:03 smithi113 ceph-mon[33540]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-05-03T19:03:04.052 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 19:03:03 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-03T19:03:04.052 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 19:03:03 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-03T19:03:04.052 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 19:03:03 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-03T19:03:04.052 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 19:03:03 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-03T19:03:04.053 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 19:03:03 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-03T19:03:04.053 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 19:03:03 smithi113 ceph-mon[33540]: from='mgr.14152 172.21.15.32:0/2141141158' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-03T19:03:04.542 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:04.541+0000 7fa619324700 1 -- 172.21.15.32:0/1993661754 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa614102fd0 msgr2=0x7fa614107020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T19:03:04.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:04.541+0000 7fa619324700 1 --2- 172.21.15.32:0/1993661754 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa614102fd0 0x7fa614107020 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fa6080099c0 tx=0x7fa608009cd0 comp rx=0 tx=0).stop 2024-05-03T19:03:04.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:04.541+0000 7fa619324700 1 -- 172.21.15.32:0/1993661754 shutdown_connections 2024-05-03T19:03:04.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:04.541+0000 7fa619324700 1 --2- 172.21.15.32:0/1993661754 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa614107700 0x7fa61410f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T19:03:04.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:04.541+0000 7fa619324700 1 --2- 172.21.15.32:0/1993661754 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa614102620 0x7fa614102a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T19:03:04.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:04.541+0000 7fa619324700 1 --2- 172.21.15.32:0/1993661754 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa614102fd0 0x7fa614107020 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T19:03:04.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:04.541+0000 7fa619324700 1 -- 172.21.15.32:0/1993661754 >> 172.21.15.32:0/1993661754 conn(0x7fa614079440 msgr2=0x7fa614079850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T19:03:04.543 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:04.541+0000 7fa619324700 1 -- 172.21.15.32:0/1993661754 shutdown_connections 2024-05-03T19:03:04.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:04.541+0000 7fa619324700 1 -- 172.21.15.32:0/1993661754 wait complete. 2024-05-03T19:03:04.544 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:04.541+0000 7fa619324700 1 Processor -- start 2024-05-03T19:03:04.545 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:04.542+0000 7fa619324700 1 -- start start 2024-05-03T19:03:04.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:04.542+0000 7fa619324700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa614102620 0x7fa61419f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T19:03:04.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:04.542+0000 7fa619324700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa614102fd0 0x7fa61419fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T19:03:04.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:04.542+0000 7fa619324700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa614107700 0x7fa614199a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T19:03:04.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:04.542+0000 7fa619324700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7fa6141125f0 con 0x7fa614102620 2024-05-03T19:03:04.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:04.542+0000 7fa619324700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa614112470 con 0x7fa614102fd0 2024-05-03T19:03:04.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:04.542+0000 7fa619324700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fa614112770 con 0x7fa614107700 2024-05-03T19:03:04.546 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:04.542+0000 7fa612ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa614102620 0x7fa61419f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T19:03:04.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:04.542+0000 7fa612ffd700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa614102620 0x7fa61419f890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56962/0 (socket says 172.21.15.32:56962) 2024-05-03T19:03:04.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:04.542+0000 7fa612ffd700 1 -- 172.21.15.32:0/1433287569 learned_addr learned my addr 172.21.15.32:0/1433287569 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T19:03:04.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:04.542+0000 7fa6127fc700 1 --2- 172.21.15.32:0/1433287569 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa614102fd0 0x7fa61419fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T19:03:04.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:04.542+0000 7fa6137fe700 1 --2- 172.21.15.32:0/1433287569 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa614107700 0x7fa614199a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T19:03:04.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:04.543+0000 7fa612ffd700 1 -- 172.21.15.32:0/1433287569 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa614107700 msgr2=0x7fa614199a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T19:03:04.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:04.543+0000 7fa612ffd700 1 --2- 172.21.15.32:0/1433287569 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa614107700 0x7fa614199a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T19:03:04.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:04.543+0000 7fa612ffd700 1 -- 172.21.15.32:0/1433287569 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa614102fd0 msgr2=0x7fa61419fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T19:03:04.547 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:04.543+0000 7fa612ffd700 1 --2- 172.21.15.32:0/1433287569 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa614102fd0 0x7fa61419fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T19:03:04.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:04.543+0000 7fa612ffd700 1 -- 172.21.15.32:0/1433287569 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa61419a1b0 con 0x7fa614102620 2024-05-03T19:03:04.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:04.543+0000 7fa6137fe700 1 --2- 172.21.15.32:0/1433287569 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa614107700 0x7fa614199a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-03T19:03:04.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:04.543+0000 7fa612ffd700 1 --2- 172.21.15.32:0/1433287569 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa614102620 0x7fa61419f890 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fa60400b530 tx=0x7fa60400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T19:03:04.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:04.543+0000 7fa5fbfff700 1 -- 172.21.15.32:0/1433287569 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa604015020 con 0x7fa614102620 2024-05-03T19:03:04.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:04.543+0000 7fa5fbfff700 1 -- 172.21.15.32:0/1433287569 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa604011ce0 con 0x7fa614102620 2024-05-03T19:03:04.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:04.543+0000 7fa619324700 1 -- 172.21.15.32:0/1433287569 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa61419a4a0 con 0x7fa614102620 2024-05-03T19:03:04.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:04.543+0000 7fa5fbfff700 1 -- 172.21.15.32:0/1433287569 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa604010e40 con 0x7fa614102620 2024-05-03T19:03:04.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:04.543+0000 7fa619324700 1 -- 172.21.15.32:0/1433287569 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6141a6760 con 0x7fa614102620 2024-05-03T19:03:04.548 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:04.544+0000 7fa619324700 1 -- 172.21.15.32:0/1433287569 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa61406a190 con 0x7fa614102620 2024-05-03T19:03:04.549 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:04.545+0000 7fa5fbfff700 1 -- 172.21.15.32:0/1433287569 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fa604018020 con 0x7fa614102620 2024-05-03T19:03:04.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:04.546+0000 7fa5fbfff700 1 --2- 172.21.15.32:0/1433287569 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fa5fc0b2990 0x7fa5fc0b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T19:03:04.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:04.546+0000 7fa6127fc700 1 --2- 172.21.15.32:0/1433287569 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fa5fc0b2990 0x7fa5fc0b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T19:03:04.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:04.546+0000 7fa5fbfff700 1 -- 172.21.15.32:0/1433287569 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8614+0+0 (secure 0 0 0) 0x7fa6040e1c10 con 0x7fa614102620 2024-05-03T19:03:04.550 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:04.547+0000 7fa6127fc700 1 --2- 172.21.15.32:0/1433287569 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fa5fc0b2990 0x7fa5fc0b4e50 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fa608006010 tx=0x7fa608005c60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T19:03:04.552 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:04.551+0000 7fa5fbfff700 1 -- 172.21.15.32:0/1433287569 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7fa60401f020 con 0x7fa614102620 2024-05-03T19:03:04.815 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:04.814+0000 7fa619324700 1 -- 172.21.15.32:0/1433287569 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fa61419af80 con 0x7fa614102620 2024-05-03T19:03:05.579 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:05.576+0000 7fa5fbfff700 1 -- 172.21.15.32:0/1433287569 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v29) ==== 104+0+0 (secure 0 0 0) 0x7fa6040a9700 con 0x7fa614102620 2024-05-03T19:03:05.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:05.579+0000 7fa619324700 1 -- 172.21.15.32:0/1433287569 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fa5fc0b2990 msgr2=0x7fa5fc0b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T19:03:05.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:05.579+0000 7fa619324700 1 --2- 172.21.15.32:0/1433287569 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fa5fc0b2990 0x7fa5fc0b4e50 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fa608006010 tx=0x7fa608005c60 comp rx=0 tx=0).stop 2024-05-03T19:03:05.581 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:05.579+0000 7fa619324700 1 -- 172.21.15.32:0/1433287569 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa614102620 msgr2=0x7fa61419f890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T19:03:05.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:05.579+0000 7fa619324700 1 --2- 172.21.15.32:0/1433287569 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa614102620 0x7fa61419f890 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fa60400b530 tx=0x7fa60400b840 comp rx=0 tx=0).stop 2024-05-03T19:03:05.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:05.579+0000 7fa619324700 1 -- 172.21.15.32:0/1433287569 shutdown_connections 2024-05-03T19:03:05.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:05.579+0000 7fa619324700 1 --2- 172.21.15.32:0/1433287569 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7fa5fc0b2990 0x7fa5fc0b4e50 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T19:03:05.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:05.579+0000 7fa619324700 1 --2- 172.21.15.32:0/1433287569 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa614107700 0x7fa614199a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T19:03:05.582 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:05.579+0000 7fa619324700 1 --2- 172.21.15.32:0/1433287569 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa614102620 0x7fa61419f890 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T19:03:05.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:05.579+0000 7fa619324700 1 --2- 172.21.15.32:0/1433287569 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa614102fd0 0x7fa61419fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T19:03:05.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:05.579+0000 7fa619324700 1 -- 172.21.15.32:0/1433287569 >> 172.21.15.32:0/1433287569 conn(0x7fa614079440 msgr2=0x7fa61406bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T19:03:05.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:05.579+0000 7fa619324700 1 -- 172.21.15.32:0/1433287569 shutdown_connections 2024-05-03T19:03:05.583 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:05.579+0000 7fa619324700 1 -- 172.21.15.32:0/1433287569 wait complete. 2024-05-03T19:03:05.593 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:03:05 smithi032 ceph-mon[29763]: pgmap v832: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 6 op/s 2024-05-03T19:03:05.593 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:03:05 smithi032 ceph-mon[29763]: from='client.? 172.21.15.32:0/1433287569' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-03T19:03:05.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 19:03:05 smithi045 ceph-mon[33299]: pgmap v832: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 6 op/s 2024-05-03T19:03:05.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 19:03:05 smithi045 ceph-mon[33299]: from='client.? 172.21.15.32:0/1433287569' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-03T19:03:06.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 19:03:05 smithi113 ceph-mon[33540]: pgmap v832: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 6 op/s 2024-05-03T19:03:06.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 19:03:05 smithi113 ceph-mon[33540]: from='client.? 172.21.15.32:0/1433287569' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-03T19:03:06.346 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid f500455e-097b-11ef-bc97-c7b262605968 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-03T19:03:06.602 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T19:03:06.869 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:03:06 smithi032 ceph-mon[29763]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-03T19:03:06.869 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:03:06 smithi032 ceph-mon[29763]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-03T19:03:06.869 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:03:06 smithi032 ceph-mon[29763]: Cluster is now healthy 2024-05-03T19:03:06.869 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:03:06 smithi032 ceph-mon[29763]: from='client.? 172.21.15.32:0/1433287569' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-05-03T19:03:06.869 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:03:06 smithi032 ceph-mon[29763]: fsmap 12 up:standby 2024-05-03T19:03:06.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 19:03:06 smithi045 ceph-mon[33299]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-03T19:03:06.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 19:03:06 smithi045 ceph-mon[33299]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-03T19:03:06.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 19:03:06 smithi045 ceph-mon[33299]: Cluster is now healthy 2024-05-03T19:03:06.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 19:03:06 smithi045 ceph-mon[33299]: from='client.? 172.21.15.32:0/1433287569' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-05-03T19:03:06.999 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 19:03:06 smithi045 ceph-mon[33299]: fsmap 12 up:standby 2024-05-03T19:03:07.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 19:03:06 smithi113 ceph-mon[33540]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-03T19:03:07.052 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 19:03:06 smithi113 ceph-mon[33540]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-03T19:03:07.052 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 19:03:06 smithi113 ceph-mon[33540]: Cluster is now healthy 2024-05-03T19:03:07.052 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 19:03:06 smithi113 ceph-mon[33540]: from='client.? 172.21.15.32:0/1433287569' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-05-03T19:03:07.052 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 19:03:06 smithi113 ceph-mon[33540]: fsmap 12 up:standby 2024-05-03T19:03:07.695 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:03:07 smithi032 ceph-mon[29763]: pgmap v833: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-05-03T19:03:07.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 19:03:07 smithi045 ceph-mon[33299]: pgmap v833: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-05-03T19:03:08.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 19:03:07 smithi113 ceph-mon[33540]: pgmap v833: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-05-03T19:03:08.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:08.225+0000 7f5103f3a700 1 -- 172.21.15.32:0/1646262541 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f50fc102fd0 msgr2=0x7f50fc107020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T19:03:08.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:08.225+0000 7f5103f3a700 1 --2- 172.21.15.32:0/1646262541 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f50fc102fd0 0x7f50fc107020 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f50f00099c0 tx=0x7f50f0009cd0 comp rx=0 tx=0).stop 2024-05-03T19:03:08.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:08.225+0000 7f5103f3a700 1 -- 172.21.15.32:0/1646262541 shutdown_connections 2024-05-03T19:03:08.228 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:08.225+0000 7f5103f3a700 1 --2- 172.21.15.32:0/1646262541 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50fc107700 0x7f50fc10f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T19:03:08.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:08.225+0000 7f5103f3a700 1 --2- 172.21.15.32:0/1646262541 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f50fc102620 0x7f50fc102a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T19:03:08.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:08.225+0000 7f5103f3a700 1 --2- 172.21.15.32:0/1646262541 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f50fc102fd0 0x7f50fc107020 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T19:03:08.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:08.225+0000 7f5103f3a700 1 -- 172.21.15.32:0/1646262541 >> 172.21.15.32:0/1646262541 conn(0x7f50fc079440 msgr2=0x7f50fc079850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T19:03:08.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:08.225+0000 7f5103f3a700 1 -- 172.21.15.32:0/1646262541 shutdown_connections 2024-05-03T19:03:08.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:08.225+0000 7f5103f3a700 1 -- 172.21.15.32:0/1646262541 wait complete. 2024-05-03T19:03:08.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:08.226+0000 7f5103f3a700 1 Processor -- start 2024-05-03T19:03:08.229 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:08.226+0000 7f5103f3a700 1 -- start start 2024-05-03T19:03:08.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:08.226+0000 7f5103f3a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50fc102620 0x7f50fc19f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T19:03:08.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:08.226+0000 7f5103f3a700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f50fc102fd0 0x7f50fc19fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T19:03:08.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:08.226+0000 7f5103f3a700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f50fc107700 0x7f50fc199a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T19:03:08.231 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:08.226+0000 7f5103f3a700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f50fc1125f0 con 0x7f50fc107700 2024-05-03T19:03:08.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:08.226+0000 7f5103f3a700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f50fc112470 con 0x7f50fc102620 2024-05-03T19:03:08.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:08.226+0000 7f5103f3a700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f50fc112770 con 0x7f50fc102fd0 2024-05-03T19:03:08.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:08.226+0000 7f51024d7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f50fc107700 0x7f50fc199a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T19:03:08.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:08.226+0000 7f51024d7700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f50fc107700 0x7f50fc199a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56992/0 (socket says 172.21.15.32:56992) 2024-05-03T19:03:08.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:08.226+0000 7f51024d7700 1 -- 172.21.15.32:0/2638833983 learned_addr learned my addr 172.21.15.32:0/2638833983 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T19:03:08.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:08.226+0000 7f5101cd6700 1 --2- 172.21.15.32:0/2638833983 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50fc102620 0x7f50fc19f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T19:03:08.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:08.226+0000 7f51014d5700 1 --2- 172.21.15.32:0/2638833983 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f50fc102fd0 0x7f50fc19fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T19:03:08.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:08.227+0000 7f51024d7700 1 -- 172.21.15.32:0/2638833983 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f50fc102fd0 msgr2=0x7f50fc19fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T19:03:08.232 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:08.227+0000 7f51024d7700 1 --2- 172.21.15.32:0/2638833983 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f50fc102fd0 0x7f50fc19fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T19:03:08.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:08.227+0000 7f51024d7700 1 -- 172.21.15.32:0/2638833983 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50fc102620 msgr2=0x7f50fc19f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T19:03:08.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:08.227+0000 7f51024d7700 1 --2- 172.21.15.32:0/2638833983 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50fc102620 0x7f50fc19f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T19:03:08.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:08.227+0000 7f51024d7700 1 -- 172.21.15.32:0/2638833983 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50fc19a1c0 con 0x7f50fc107700 2024-05-03T19:03:08.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:08.227+0000 7f51024d7700 1 --2- 172.21.15.32:0/2638833983 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f50fc107700 0x7f50fc199a60 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f50f400b550 tx=0x7f50f400b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T19:03:08.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:08.227+0000 7f50faffd700 1 -- 172.21.15.32:0/2638833983 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50f4015020 con 0x7f50fc107700 2024-05-03T19:03:08.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:08.227+0000 7f50faffd700 1 -- 172.21.15.32:0/2638833983 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50f4011ce0 con 0x7f50fc107700 2024-05-03T19:03:08.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:08.227+0000 7f5103f3a700 1 -- 172.21.15.32:0/2638833983 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50fc19a4b0 con 0x7f50fc107700 2024-05-03T19:03:08.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:08.227+0000 7f5103f3a700 1 -- 172.21.15.32:0/2638833983 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50fc103ff0 con 0x7f50fc107700 2024-05-03T19:03:08.233 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:08.227+0000 7f50faffd700 1 -- 172.21.15.32:0/2638833983 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50f4010c90 con 0x7f50fc107700 2024-05-03T19:03:08.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:08.228+0000 7f5103f3a700 1 -- 172.21.15.32:0/2638833983 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50fc06a190 con 0x7f50fc107700 2024-05-03T19:03:08.234 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:08.230+0000 7f50faffd700 1 -- 172.21.15.32:0/2638833983 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f50f4018020 con 0x7f50fc107700 2024-05-03T19:03:08.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:08.230+0000 7f50faffd700 1 --2- 172.21.15.32:0/2638833983 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f50e40b2990 0x7f50e40b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T19:03:08.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:08.231+0000 7f5101cd6700 1 --2- 172.21.15.32:0/2638833983 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f50e40b2990 0x7f50e40b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T19:03:08.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:08.231+0000 7f50faffd700 1 -- 172.21.15.32:0/2638833983 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8614+0+0 (secure 0 0 0) 0x7f50f40e2020 con 0x7f50fc107700 2024-05-03T19:03:08.235 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:08.231+0000 7f5101cd6700 1 --2- 172.21.15.32:0/2638833983 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f50e40b2990 0x7f50e40b4e50 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f50ec009550 tx=0x7f50ec0091b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T19:03:08.236 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:08.234+0000 7f50faffd700 1 -- 172.21.15.32:0/2638833983 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f50f401f020 con 0x7f50fc107700 2024-05-03T19:03:08.452 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:08.449+0000 7f5103f3a700 1 -- 172.21.15.32:0/2638833983 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f50fc02e510 con 0x7f50fc107700 2024-05-03T19:03:08.594 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:08.592+0000 7f50faffd700 1 -- 172.21.15.32:0/2638833983 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v84) ==== 181+0+0 (secure 0 0 0) 0x7f50f40a9b10 con 0x7f50fc107700 2024-05-03T19:03:08.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:08.595+0000 7f5103f3a700 1 -- 172.21.15.32:0/2638833983 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f50e40b2990 msgr2=0x7f50e40b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T19:03:08.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:08.595+0000 7f5103f3a700 1 --2- 172.21.15.32:0/2638833983 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f50e40b2990 0x7f50e40b4e50 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f50ec009550 tx=0x7f50ec0091b0 comp rx=0 tx=0).stop 2024-05-03T19:03:08.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:08.595+0000 7f5103f3a700 1 -- 172.21.15.32:0/2638833983 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f50fc107700 msgr2=0x7f50fc199a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T19:03:08.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:08.595+0000 7f5103f3a700 1 --2- 172.21.15.32:0/2638833983 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f50fc107700 0x7f50fc199a60 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f50f400b550 tx=0x7f50f400b860 comp rx=0 tx=0).stop 2024-05-03T19:03:08.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:08.595+0000 7f5103f3a700 1 -- 172.21.15.32:0/2638833983 shutdown_connections 2024-05-03T19:03:08.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:08.595+0000 7f5103f3a700 1 --2- 172.21.15.32:0/2638833983 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f50e40b2990 0x7f50e40b4e50 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T19:03:08.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:08.595+0000 7f5103f3a700 1 --2- 172.21.15.32:0/2638833983 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f50fc107700 0x7f50fc199a60 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T19:03:08.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:08.596+0000 7f5103f3a700 1 --2- 172.21.15.32:0/2638833983 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f50fc102620 0x7f50fc19f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T19:03:08.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:08.596+0000 7f5103f3a700 1 --2- 172.21.15.32:0/2638833983 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f50fc102fd0 0x7f50fc19fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T19:03:08.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:08.596+0000 7f5103f3a700 1 -- 172.21.15.32:0/2638833983 >> 172.21.15.32:0/2638833983 conn(0x7f50fc079440 msgr2=0x7f50fc06bed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T19:03:08.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:08.596+0000 7f5103f3a700 1 -- 172.21.15.32:0/2638833983 shutdown_connections 2024-05-03T19:03:08.599 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:08.596+0000 7f5103f3a700 1 -- 172.21.15.32:0/2638833983 wait complete. 2024-05-03T19:03:08.600 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_metadata' removed 2024-05-03T19:03:08.944 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:03:08 smithi032 ceph-mon[29763]: from='client.? 172.21.15.32:0/2638833983' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-03T19:03:08.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 19:03:08 smithi045 ceph-mon[33299]: from='client.? 172.21.15.32:0/2638833983' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-03T19:03:09.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 19:03:08 smithi113 ceph-mon[33540]: from='client.? 172.21.15.32:0/2638833983' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-03T19:03:09.226 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea shell --fsid f500455e-097b-11ef-bc97-c7b262605968 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-03T19:03:09.475 INFO:teuthology.orchestra.run.smithi032.stderr:Inferring config /var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/mon.a/config 2024-05-03T19:03:09.848 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:03:09 smithi032 ceph-mon[29763]: pgmap v834: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-03T19:03:09.848 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:03:09 smithi032 ceph-mon[29763]: from='client.? 172.21.15.32:0/2638833983' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-05-03T19:03:09.848 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:03:09 smithi032 ceph-mon[29763]: osdmap e84: 12 total, 12 up, 12 in 2024-05-03T19:03:09.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 19:03:09 smithi045 ceph-mon[33299]: pgmap v834: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-03T19:03:09.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 19:03:09 smithi045 ceph-mon[33299]: from='client.? 172.21.15.32:0/2638833983' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-05-03T19:03:09.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 19:03:09 smithi045 ceph-mon[33299]: osdmap e84: 12 total, 12 up, 12 in 2024-05-03T19:03:10.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 19:03:09 smithi113 ceph-mon[33540]: pgmap v834: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 3 op/s 2024-05-03T19:03:10.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 19:03:09 smithi113 ceph-mon[33540]: from='client.? 172.21.15.32:0/2638833983' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-05-03T19:03:10.052 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 19:03:09 smithi113 ceph-mon[33540]: osdmap e84: 12 total, 12 up, 12 in 2024-05-03T19:03:11.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:11.102+0000 7f3bf4d52700 1 -- 172.21.15.32:0/1138645506 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3bf0108850 msgr2=0x7f3bf01103d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T19:03:11.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:11.102+0000 7f3bf4d52700 1 --2- 172.21.15.32:0/1138645506 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3bf0108850 0x7f3bf01103d0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f3be800b0d0 tx=0x7f3be800b3e0 comp rx=0 tx=0).stop 2024-05-03T19:03:11.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:11.102+0000 7f3bf4d52700 1 -- 172.21.15.32:0/1138645506 shutdown_connections 2024-05-03T19:03:11.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:11.102+0000 7f3bf4d52700 1 --2- 172.21.15.32:0/1138645506 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3bf0108850 0x7f3bf01103d0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T19:03:11.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:11.102+0000 7f3bf4d52700 1 --2- 172.21.15.32:0/1138645506 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3bf0103750 0x7f3bf0103b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T19:03:11.108 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:11.102+0000 7f3bf4d52700 1 --2- 172.21.15.32:0/1138645506 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3bf0104100 0x7f3bf0108120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T19:03:11.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:11.102+0000 7f3bf4d52700 1 -- 172.21.15.32:0/1138645506 >> 172.21.15.32:0/1138645506 conn(0x7f3bf007a600 msgr2=0x7f3bf007aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T19:03:11.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:11.102+0000 7f3bf4d52700 1 -- 172.21.15.32:0/1138645506 shutdown_connections 2024-05-03T19:03:11.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:11.102+0000 7f3bf4d52700 1 -- 172.21.15.32:0/1138645506 wait complete. 2024-05-03T19:03:11.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:11.103+0000 7f3bf4d52700 1 Processor -- start 2024-05-03T19:03:11.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:11.103+0000 7f3bf4d52700 1 -- start start 2024-05-03T19:03:11.109 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:11.103+0000 7f3bf4d52700 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3bf0103750 0x7f3bf019f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T19:03:11.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:11.103+0000 7f3bf4d52700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3bf0104100 0x7f3bf019fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T19:03:11.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:11.103+0000 7f3bf4d52700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3bf0108850 0x7f3bf01997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T19:03:11.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:11.103+0000 7f3bf4d52700 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 -- 0x7f3bf0113670 con 0x7f3bf0108850 2024-05-03T19:03:11.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:11.103+0000 7f3bf4d52700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3bf01134f0 con 0x7f3bf0104100 2024-05-03T19:03:11.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:11.103+0000 7f3bf4d52700 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f3bf01137f0 con 0x7f3bf0103750 2024-05-03T19:03:11.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:11.103+0000 7f3beed9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3bf0108850 0x7f3bf01997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T19:03:11.110 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:11.104+0000 7f3beed9d700 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3bf0108850 0x7f3bf01997d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57016/0 (socket says 172.21.15.32:57016) 2024-05-03T19:03:11.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:11.104+0000 7f3beed9d700 1 -- 172.21.15.32:0/1345632033 learned_addr learned my addr 172.21.15.32:0/1345632033 (peer_addr_for_me v2:172.21.15.32:0/0) 2024-05-03T19:03:11.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:11.104+0000 7f3bee59c700 1 --2- 172.21.15.32:0/1345632033 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3bf0103750 0x7f3bf019f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T19:03:11.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:11.104+0000 7f3be7fff700 1 --2- 172.21.15.32:0/1345632033 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3bf0104100 0x7f3bf019fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T19:03:11.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:11.104+0000 7f3beed9d700 1 -- 172.21.15.32:0/1345632033 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3bf0103750 msgr2=0x7f3bf019f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T19:03:11.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:11.104+0000 7f3beed9d700 1 --2- 172.21.15.32:0/1345632033 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3bf0103750 0x7f3bf019f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T19:03:11.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:11.104+0000 7f3beed9d700 1 -- 172.21.15.32:0/1345632033 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3bf0104100 msgr2=0x7f3bf019fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T19:03:11.111 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:11.104+0000 7f3beed9d700 1 --2- 172.21.15.32:0/1345632033 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3bf0104100 0x7f3bf019fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T19:03:11.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:11.104+0000 7f3beed9d700 1 -- 172.21.15.32:0/1345632033 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bf0199fb0 con 0x7f3bf0108850 2024-05-03T19:03:11.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:11.104+0000 7f3beed9d700 1 --2- 172.21.15.32:0/1345632033 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3bf0108850 0x7f3bf01997d0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f3be80079a0 tx=0x7f3be80079d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T19:03:11.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:11.104+0000 7f3be77fe700 1 -- 172.21.15.32:0/1345632033 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3be801f070 con 0x7f3bf0108850 2024-05-03T19:03:11.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:11.104+0000 7f3bf4d52700 1 -- 172.21.15.32:0/1345632033 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bf019a240 con 0x7f3bf0108850 2024-05-03T19:03:11.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:11.104+0000 7f3be77fe700 1 -- 172.21.15.32:0/1345632033 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3be8003c00 con 0x7f3bf0108850 2024-05-03T19:03:11.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:11.104+0000 7f3be77fe700 1 -- 172.21.15.32:0/1345632033 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3be801b6a0 con 0x7f3bf0108850 2024-05-03T19:03:11.112 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:11.104+0000 7f3bf4d52700 1 -- 172.21.15.32:0/1345632033 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bf01a64b0 con 0x7f3bf0108850 2024-05-03T19:03:11.113 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:11.105+0000 7f3bf4d52700 1 -- 172.21.15.32:0/1345632033 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3bf0104be0 con 0x7f3bf0108850 2024-05-03T19:03:11.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:11.107+0000 7f3be77fe700 1 -- 172.21.15.32:0/1345632033 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f3be8022070 con 0x7f3bf0108850 2024-05-03T19:03:11.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:11.108+0000 7f3be77fe700 1 --2- 172.21.15.32:0/1345632033 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f3bdc0b2910 0x7f3bdc0b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-03T19:03:11.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:11.108+0000 7f3be77fe700 1 -- 172.21.15.32:0/1345632033 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8060+0+0 (secure 0 0 0) 0x7f3be80e3450 con 0x7f3bf0108850 2024-05-03T19:03:11.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:11.108+0000 7f3bee59c700 1 --2- 172.21.15.32:0/1345632033 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f3bdc0b2910 0x7f3bdc0b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-03T19:03:11.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:11.108+0000 7f3bee59c700 1 --2- 172.21.15.32:0/1345632033 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f3bdc0b2910 0x7f3bdc0b4dd0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f3be000ac50 tx=0x7f3be000a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-03T19:03:11.114 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:11.111+0000 7f3be77fe700 1 -- 172.21.15.32:0/1345632033 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191839 (secure 0 0 0) 0x7f3be80ab200 con 0x7f3bf0108850 2024-05-03T19:03:11.328 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:11.326+0000 7f3bf4d52700 1 -- 172.21.15.32:0/1345632033 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f3bf01a6680 con 0x7f3bf0108850 2024-05-03T19:03:11.598 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:11.597+0000 7f3be77fe700 1 -- 172.21.15.32:0/1345632033 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v85) ==== 169+0+0 (secure 0 0 0) 0x7f3be80aab00 con 0x7f3bf0108850 2024-05-03T19:03:11.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:11.600+0000 7f3bf4d52700 1 -- 172.21.15.32:0/1345632033 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f3bdc0b2910 msgr2=0x7f3bdc0b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T19:03:11.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:11.600+0000 7f3bf4d52700 1 --2- 172.21.15.32:0/1345632033 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f3bdc0b2910 0x7f3bdc0b4dd0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f3be000ac50 tx=0x7f3be000a380 comp rx=0 tx=0).stop 2024-05-03T19:03:11.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:11.600+0000 7f3bf4d52700 1 -- 172.21.15.32:0/1345632033 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3bf0108850 msgr2=0x7f3bf01997d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-03T19:03:11.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:11.600+0000 7f3bf4d52700 1 --2- 172.21.15.32:0/1345632033 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3bf0108850 0x7f3bf01997d0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f3be80079a0 tx=0x7f3be80079d0 comp rx=0 tx=0).stop 2024-05-03T19:03:11.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:11.600+0000 7f3bf4d52700 1 -- 172.21.15.32:0/1345632033 shutdown_connections 2024-05-03T19:03:11.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:11.600+0000 7f3bf4d52700 1 --2- 172.21.15.32:0/1345632033 >> [v2:172.21.15.32:6800/1796547904,v1:172.21.15.32:6801/1796547904] conn(0x7f3bdc0b2910 0x7f3bdc0b4dd0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T19:03:11.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:11.600+0000 7f3bf4d52700 1 --2- 172.21.15.32:0/1345632033 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3bf0108850 0x7f3bf01997d0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T19:03:11.603 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:11.600+0000 7f3bf4d52700 1 --2- 172.21.15.32:0/1345632033 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3bf0103750 0x7f3bf019f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T19:03:11.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:11.600+0000 7f3bf4d52700 1 --2- 172.21.15.32:0/1345632033 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3bf0104100 0x7f3bf019fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-03T19:03:11.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:11.600+0000 7f3bf4d52700 1 -- 172.21.15.32:0/1345632033 >> 172.21.15.32:0/1345632033 conn(0x7f3bf007a600 msgr2=0x7f3bf01013c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-03T19:03:11.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:11.601+0000 7f3bf4d52700 1 -- 172.21.15.32:0/1345632033 shutdown_connections 2024-05-03T19:03:11.604 INFO:teuthology.orchestra.run.smithi032.stderr:2024-05-03T19:03:11.601+0000 7f3bf4d52700 1 -- 172.21.15.32:0/1345632033 wait complete. 2024-05-03T19:03:11.605 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_data' removed 2024-05-03T19:03:11.950 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:03:11 smithi032 ceph-mon[29763]: pgmap v836: 65 pgs: 65 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T19:03:11.950 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:03:11 smithi032 ceph-mon[29763]: from='client.? 172.21.15.32:0/1345632033' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-03T19:03:11.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 19:03:11 smithi045 ceph-mon[33299]: pgmap v836: 65 pgs: 65 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T19:03:11.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 19:03:11 smithi045 ceph-mon[33299]: from='client.? 172.21.15.32:0/1345632033' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-03T19:03:12.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 19:03:11 smithi113 ceph-mon[33540]: pgmap v836: 65 pgs: 65 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-03T19:03:12.051 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 19:03:11 smithi113 ceph-mon[33540]: from='client.? 172.21.15.32:0/1345632033' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-03T19:03:12.282 DEBUG:teuthology.orchestra.run.smithi032:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-03T19:03:12.312 DEBUG:teuthology.orchestra.run.smithi045:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-03T19:03:12.345 DEBUG:teuthology.orchestra.run.smithi113:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-03T19:03:12.377 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-05-03T19:03:12.377 DEBUG:teuthology.orchestra.run.smithi032:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-03T19:03:12.394 DEBUG:teuthology.orchestra.run.smithi045:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-03T19:03:12.411 DEBUG:teuthology.orchestra.run.smithi113:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-03T19:03:12.435 INFO:tasks.cephadm:Stopping all daemons... 2024-05-03T19:03:12.435 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-05-03T19:03:12.435 DEBUG:teuthology.orchestra.run.smithi032:> sudo systemctl stop ceph-f500455e-097b-11ef-bc97-c7b262605968@mon.a 2024-05-03T19:03:12.569 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:03:12 smithi032 systemd[1]: Stopping Ceph mon.a for f500455e-097b-11ef-bc97-c7b262605968... 2024-05-03T19:03:12.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 19:03:12 smithi113 ceph-mon[33540]: from='client.? 172.21.15.32:0/1345632033' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-05-03T19:03:12.802 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 19:03:12 smithi113 ceph-mon[33540]: osdmap e85: 12 total, 12 up, 12 in 2024-05-03T19:03:12.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 19:03:12 smithi045 ceph-mon[33299]: from='client.? 172.21.15.32:0/1345632033' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-05-03T19:03:12.998 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 19:03:12 smithi045 ceph-mon[33299]: osdmap e85: 12 total, 12 up, 12 in 2024-05-03T19:03:13.046 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:03:12 smithi032 ceph-mon[29763]: from='client.? 172.21.15.32:0/1345632033' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-05-03T19:03:13.047 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:03:12 smithi032 ceph-mon[29763]: osdmap e85: 12 total, 12 up, 12 in 2024-05-03T19:03:13.047 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:03:12 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-mon-a[29759]: 2024-05-03T19:03:12.727+0000 7f050da4e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-03T19:03:13.047 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:03:12 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-mon-a[29759]: 2024-05-03T19:03:12.727+0000 7f050da4e700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-05-03T19:03:13.047 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:03:12 smithi032 podman[87280]: 2024-05-03 19:03:12.779544825 +0000 UTC m=+0.151580621 container died b018994d5361d5479c162f8af535e5ecc6199a8c32a7196b0b9a5c86d802daf6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240430, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-03T19:03:13.313 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:03:13 smithi032 podman[87280]: 2024-05-03 19:03:13.128085035 +0000 UTC m=+0.500120840 container cleanup b018994d5361d5479c162f8af535e5ecc6199a8c32a7196b0b9a5c86d802daf6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-mon-a, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240430, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-03T19:03:13.313 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:03:13 smithi032 bash[87280]: ceph-f500455e-097b-11ef-bc97-c7b262605968-mon-a 2024-05-03T19:03:13.555 DEBUG:teuthology.orchestra.run.smithi032:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f500455e-097b-11ef-bc97-c7b262605968@mon.a.service' 2024-05-03T19:03:13.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:03:13 smithi032 podman[87294]: 2024-05-03 19:03:13.312709819 +0000 UTC m=+0.521690631 container remove b018994d5361d5479c162f8af535e5ecc6199a8c32a7196b0b9a5c86d802daf6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-mon-a, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240430, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-05-03T19:03:13.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:03:13 smithi032 systemd[1]: ceph-f500455e-097b-11ef-bc97-c7b262605968@mon.a.service: Deactivated successfully. 2024-05-03T19:03:13.568 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:03:13 smithi032 systemd[1]: Stopped Ceph mon.a for f500455e-097b-11ef-bc97-c7b262605968. 2024-05-03T19:03:13.569 INFO:journalctl@ceph.mon.a.smithi032.stdout:May 03 19:03:13 smithi032 systemd[1]: ceph-f500455e-097b-11ef-bc97-c7b262605968@mon.a.service: Consumed 20.947s CPU time. 2024-05-03T19:03:13.599 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-03T19:03:13.600 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-05-03T19:03:13.600 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-05-03T19:03:13.600 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ceph-f500455e-097b-11ef-bc97-c7b262605968@mon.b 2024-05-03T19:03:13.909 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 19:03:13 smithi045 systemd[1]: Stopping Ceph mon.b for f500455e-097b-11ef-bc97-c7b262605968... 2024-05-03T19:03:14.248 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 19:03:13 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-mon-b[33295]: 2024-05-03T19:03:13.907+0000 7f99d937c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-03T19:03:14.248 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 19:03:13 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-mon-b[33295]: 2024-05-03T19:03:13.907+0000 7f99d937c700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-05-03T19:03:14.248 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 19:03:14 smithi045 podman[68217]: 2024-05-03 19:03:14.143749545 +0000 UTC m=+0.338857442 container died 8d050076af5619b684ab893d24573c7cb63577bae88639decd2c9150867584ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-mon-b, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240430, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-03T19:03:14.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 19:03:14 smithi045 podman[68217]: 2024-05-03 19:03:14.616538269 +0000 UTC m=+0.811646182 container cleanup 8d050076af5619b684ab893d24573c7cb63577bae88639decd2c9150867584ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-mon-b, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240430, org.label-schema.schema-version=1.0) 2024-05-03T19:03:14.748 INFO:journalctl@ceph.mon.b.smithi045.stdout:May 03 19:03:14 smithi045 bash[68217]: ceph-f500455e-097b-11ef-bc97-c7b262605968-mon-b 2024-05-03T19:03:15.004 DEBUG:teuthology.orchestra.run.smithi045:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f500455e-097b-11ef-bc97-c7b262605968@mon.b.service' 2024-05-03T19:03:15.050 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-03T19:03:15.050 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-05-03T19:03:15.050 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-05-03T19:03:15.050 DEBUG:teuthology.orchestra.run.smithi113:> sudo systemctl stop ceph-f500455e-097b-11ef-bc97-c7b262605968@mon.c 2024-05-03T19:03:15.391 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 19:03:15 smithi113 systemd[1]: Stopping Ceph mon.c for f500455e-097b-11ef-bc97-c7b262605968... 2024-05-03T19:03:15.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 19:03:15 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-mon-c[33536]: 2024-05-03T19:03:15.388+0000 7fad1f462700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-03T19:03:15.801 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 19:03:15 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-mon-c[33536]: 2024-05-03T19:03:15.388+0000 7fad1f462700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-05-03T19:03:15.802 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 19:03:15 smithi113 podman[69805]: 2024-05-03 19:03:15.584341533 +0000 UTC m=+0.340058678 container died c64669fc044e19e5ecac92044b592f702fc2628cf33921c398f05947fad37a96 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-mon-c, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240430, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe) 2024-05-03T19:03:16.301 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 19:03:16 smithi113 podman[69805]: 2024-05-03 19:03:16.049474776 +0000 UTC m=+0.805191972 container cleanup c64669fc044e19e5ecac92044b592f702fc2628cf33921c398f05947fad37a96 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-mon-c, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240430, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-03T19:03:16.302 INFO:journalctl@ceph.mon.c.smithi113.stdout:May 03 19:03:16 smithi113 bash[69805]: ceph-f500455e-097b-11ef-bc97-c7b262605968-mon-c 2024-05-03T19:03:16.485 DEBUG:teuthology.orchestra.run.smithi113:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f500455e-097b-11ef-bc97-c7b262605968@mon.c.service' 2024-05-03T19:03:16.534 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-03T19:03:16.535 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-05-03T19:03:16.535 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-05-03T19:03:16.535 DEBUG:teuthology.orchestra.run.smithi032:> sudo systemctl stop ceph-f500455e-097b-11ef-bc97-c7b262605968@mgr.x 2024-05-03T19:03:16.818 INFO:journalctl@ceph.mgr.x.smithi032.stdout:May 03 19:03:16 smithi032 systemd[1]: Stopping Ceph mgr.x for f500455e-097b-11ef-bc97-c7b262605968... 2024-05-03T19:03:17.184 INFO:journalctl@ceph.mgr.x.smithi032.stdout:May 03 19:03:16 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-x[29995]: 2024-05-03T19:03:16.848+0000 7fed2b782700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-03T19:03:17.185 INFO:journalctl@ceph.mgr.x.smithi032.stdout:May 03 19:03:16 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-x[29995]: 2024-05-03T19:03:16.848+0000 7fed2b782700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-05-03T19:03:17.185 INFO:journalctl@ceph.mgr.x.smithi032.stdout:May 03 19:03:16 smithi032 podman[87402]: 2024-05-03 19:03:16.880357714 +0000 UTC m=+0.149278968 container died bc042bf2e9b3d95b425de1863e57f1ae61ece100852b9f123efdb71ecd143517 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-x, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240430, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-05-03T19:03:17.509 INFO:journalctl@ceph.mgr.x.smithi032.stdout:May 03 19:03:17 smithi032 podman[87402]: 2024-05-03 19:03:17.28262747 +0000 UTC m=+0.551548707 container cleanup bc042bf2e9b3d95b425de1863e57f1ae61ece100852b9f123efdb71ecd143517 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9a047fe7599a549042d06a3fb5b6813a203eabea, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-x, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.build-date=20240430, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-05-03T19:03:17.510 INFO:journalctl@ceph.mgr.x.smithi032.stdout:May 03 19:03:17 smithi032 bash[87402]: ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-x 2024-05-03T19:03:17.615 DEBUG:teuthology.orchestra.run.smithi032:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f500455e-097b-11ef-bc97-c7b262605968@mgr.x.service' 2024-05-03T19:03:17.651 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-03T19:03:17.652 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-05-03T19:03:17.652 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-05-03T19:03:17.652 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ceph-f500455e-097b-11ef-bc97-c7b262605968@mgr.y 2024-05-03T19:03:17.976 INFO:journalctl@ceph.mgr.y.smithi045.stdout:May 03 19:03:17 smithi045 systemd[1]: Stopping Ceph mgr.y for f500455e-097b-11ef-bc97-c7b262605968... 2024-05-03T19:03:18.248 INFO:journalctl@ceph.mgr.y.smithi045.stdout:May 03 19:03:17 smithi045 podman[68341]: 2024-05-03 19:03:17.975235859 +0000 UTC m=+0.121829801 container died 57afcca919d642837a6928130b4ac0371edc1da73a012565916cb6afaf3579a2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-y, ceph=True, org.label-schema.build-date=20240430, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-05-03T19:03:18.615 INFO:journalctl@ceph.mgr.y.smithi045.stdout:May 03 19:03:18 smithi045 podman[68341]: 2024-05-03 19:03:18.387852299 +0000 UTC m=+0.534446230 container cleanup 57afcca919d642837a6928130b4ac0371edc1da73a012565916cb6afaf3579a2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-y, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240430, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-05-03T19:03:18.615 INFO:journalctl@ceph.mgr.y.smithi045.stdout:May 03 19:03:18 smithi045 bash[68341]: ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-y 2024-05-03T19:03:18.718 DEBUG:teuthology.orchestra.run.smithi045:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f500455e-097b-11ef-bc97-c7b262605968@mgr.y.service' 2024-05-03T19:03:18.753 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-03T19:03:18.753 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-05-03T19:03:18.753 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-05-03T19:03:18.753 DEBUG:teuthology.orchestra.run.smithi113:> sudo systemctl stop ceph-f500455e-097b-11ef-bc97-c7b262605968@mgr.z 2024-05-03T19:03:19.051 INFO:journalctl@ceph.mgr.z.smithi113.stdout:May 03 19:03:18 smithi113 systemd[1]: Stopping Ceph mgr.z for f500455e-097b-11ef-bc97-c7b262605968... 2024-05-03T19:03:19.381 INFO:journalctl@ceph.mgr.z.smithi113.stdout:May 03 19:03:19 smithi113 podman[69929]: 2024-05-03 19:03:19.08388475 +0000 UTC m=+0.140368749 container died 5dbb8da288720641a1a2cf3f69b36e77246af4204687fd3dd426a3d2c60ac87e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-z, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.build-date=20240430) 2024-05-03T19:03:19.726 INFO:journalctl@ceph.mgr.z.smithi113.stdout:May 03 19:03:19 smithi113 podman[69929]: 2024-05-03 19:03:19.457336968 +0000 UTC m=+0.513820970 container cleanup 5dbb8da288720641a1a2cf3f69b36e77246af4204687fd3dd426a3d2c60ac87e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-z, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240430, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe) 2024-05-03T19:03:19.726 INFO:journalctl@ceph.mgr.z.smithi113.stdout:May 03 19:03:19 smithi113 bash[69929]: ceph-f500455e-097b-11ef-bc97-c7b262605968-mgr-z 2024-05-03T19:03:19.886 DEBUG:teuthology.orchestra.run.smithi113:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f500455e-097b-11ef-bc97-c7b262605968@mgr.z.service' 2024-05-03T19:03:19.920 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-03T19:03:19.920 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-05-03T19:03:19.921 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-05-03T19:03:19.921 DEBUG:teuthology.orchestra.run.smithi032:> sudo systemctl stop ceph-f500455e-097b-11ef-bc97-c7b262605968@osd.0 2024-05-03T19:03:20.220 INFO:journalctl@ceph.osd.0.smithi032.stdout:May 03 19:03:19 smithi032 systemd[1]: Stopping Ceph osd.0 for f500455e-097b-11ef-bc97-c7b262605968... 2024-05-03T19:03:20.568 INFO:journalctl@ceph.osd.0.smithi032.stdout:May 03 19:03:20 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-0[42356]: 2024-05-03T19:03:20.219+0000 7f6e8257f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-03T19:03:20.568 INFO:journalctl@ceph.osd.0.smithi032.stdout:May 03 19:03:20 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-0[42356]: 2024-05-03T19:03:20.219+0000 7f6e8257f700 -1 osd.0 85 *** Got signal Terminated *** 2024-05-03T19:03:20.569 INFO:journalctl@ceph.osd.0.smithi032.stdout:May 03 19:03:20 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-0[42356]: 2024-05-03T19:03:20.219+0000 7f6e8257f700 -1 osd.0 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-03T19:03:25.956 INFO:journalctl@ceph.osd.0.smithi032.stdout:May 03 19:03:25 smithi032 podman[87523]: 2024-05-03 19:03:25.649922266 +0000 UTC m=+5.530985757 container died f20a1672f110098f4214ad3196768c03711c984d5e2707e512b4516b24c2a81a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240430) 2024-05-03T19:03:26.297 INFO:journalctl@ceph.osd.0.smithi032.stdout:May 03 19:03:26 smithi032 podman[87523]: 2024-05-03 19:03:26.04529645 +0000 UTC m=+5.926359936 container cleanup f20a1672f110098f4214ad3196768c03711c984d5e2707e512b4516b24c2a81a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-0, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, io.buildah.version=1.33.5, org.label-schema.build-date=20240430, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-03T19:03:26.297 INFO:journalctl@ceph.osd.0.smithi032.stdout:May 03 19:03:26 smithi032 bash[87523]: ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-0 2024-05-03T19:03:26.568 INFO:journalctl@ceph.osd.0.smithi032.stdout:May 03 19:03:26 smithi032 podman[87543]: 2024-05-03 19:03:26.296280403 +0000 UTC m=+0.637543423 container remove f20a1672f110098f4214ad3196768c03711c984d5e2707e512b4516b24c2a81a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-0, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240430) 2024-05-03T19:03:26.933 INFO:journalctl@ceph.osd.0.smithi032.stdout:May 03 19:03:26 smithi032 podman[87633]: 2024-05-03 19:03:26.61393773 +0000 UTC m=+0.110755356 container create f8b49af183f97c26ef6188cc9e267c6ac41ef84accef89dda27b2c45d6c23edd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-0-deactivate, org.label-schema.build-date=20240430, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-05-03T19:03:26.934 INFO:journalctl@ceph.osd.0.smithi032.stdout:May 03 19:03:26 smithi032 podman[87633]: 2024-05-03 19:03:26.521368657 +0000 UTC m=+0.018186307 image pull ee4d846202496ba6096593e8e8becc0700a5eb7ebd95536031187f5f337d4012 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd 2024-05-03T19:03:26.934 INFO:journalctl@ceph.osd.0.smithi032.stdout:May 03 19:03:26 smithi032 podman[87633]: 2024-05-03 19:03:26.857947264 +0000 UTC m=+0.354764888 container init f8b49af183f97c26ef6188cc9e267c6ac41ef84accef89dda27b2c45d6c23edd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-0-deactivate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240430) 2024-05-03T19:03:26.934 INFO:journalctl@ceph.osd.0.smithi032.stdout:May 03 19:03:26 smithi032 podman[87633]: 2024-05-03 19:03:26.861305466 +0000 UTC m=+0.358123094 container start f8b49af183f97c26ef6188cc9e267c6ac41ef84accef89dda27b2c45d6c23edd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-0-deactivate, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240430, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-03T19:03:26.934 INFO:journalctl@ceph.osd.0.smithi032.stdout:May 03 19:03:26 smithi032 podman[87633]: 2024-05-03 19:03:26.932699508 +0000 UTC m=+0.429517132 container attach f8b49af183f97c26ef6188cc9e267c6ac41ef84accef89dda27b2c45d6c23edd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-0-deactivate, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240430, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe) 2024-05-03T19:03:27.319 INFO:journalctl@ceph.osd.0.smithi032.stdout:May 03 19:03:27 smithi032 podman[87633]: 2024-05-03 19:03:27.166577438 +0000 UTC m=+0.663395070 container died f8b49af183f97c26ef6188cc9e267c6ac41ef84accef89dda27b2c45d6c23edd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-0-deactivate, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240430) 2024-05-03T19:03:27.729 DEBUG:teuthology.orchestra.run.smithi032:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f500455e-097b-11ef-bc97-c7b262605968@osd.0.service' 2024-05-03T19:03:27.769 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-03T19:03:27.770 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-05-03T19:03:27.770 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-05-03T19:03:27.770 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ceph-f500455e-097b-11ef-bc97-c7b262605968@osd.1 2024-05-03T19:03:28.059 INFO:journalctl@ceph.osd.1.smithi045.stdout:May 03 19:03:27 smithi045 systemd[1]: Stopping Ceph osd.1 for f500455e-097b-11ef-bc97-c7b262605968... 2024-05-03T19:03:28.498 INFO:journalctl@ceph.osd.1.smithi045.stdout:May 03 19:03:28 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-1[37891]: 2024-05-03T19:03:28.057+0000 7f87e15be700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-03T19:03:28.499 INFO:journalctl@ceph.osd.1.smithi045.stdout:May 03 19:03:28 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-1[37891]: 2024-05-03T19:03:28.057+0000 7f87e15be700 -1 osd.1 85 *** Got signal Terminated *** 2024-05-03T19:03:28.499 INFO:journalctl@ceph.osd.1.smithi045.stdout:May 03 19:03:28 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-1[37891]: 2024-05-03T19:03:28.057+0000 7f87e15be700 -1 osd.1 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-03T19:03:33.748 INFO:journalctl@ceph.osd.1.smithi045.stdout:May 03 19:03:33 smithi045 podman[68466]: 2024-05-03 19:03:33.394677574 +0000 UTC m=+5.425611805 container died 94b5ca8dfd0113a611b48682141b28ca1d67f356f6304ba92cf67f1d84d63555 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-1, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240430, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-03T19:03:34.135 INFO:journalctl@ceph.osd.1.smithi045.stdout:May 03 19:03:33 smithi045 podman[68466]: 2024-05-03 19:03:33.88362922 +0000 UTC m=+5.914563450 container cleanup 94b5ca8dfd0113a611b48682141b28ca1d67f356f6304ba92cf67f1d84d63555 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-1, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.build-date=20240430, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-05-03T19:03:34.136 INFO:journalctl@ceph.osd.1.smithi045.stdout:May 03 19:03:33 smithi045 bash[68466]: ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-1 2024-05-03T19:03:34.458 INFO:journalctl@ceph.osd.1.smithi045.stdout:May 03 19:03:34 smithi045 podman[68494]: 2024-05-03 19:03:34.135132201 +0000 UTC m=+0.735741969 container remove 94b5ca8dfd0113a611b48682141b28ca1d67f356f6304ba92cf67f1d84d63555 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-1, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240430, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-05-03T19:03:34.748 INFO:journalctl@ceph.osd.1.smithi045.stdout:May 03 19:03:34 smithi045 podman[68584]: 2024-05-03 19:03:34.361450616 +0000 UTC m=+0.018815920 image pull ee4d846202496ba6096593e8e8becc0700a5eb7ebd95536031187f5f337d4012 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd 2024-05-03T19:03:34.748 INFO:journalctl@ceph.osd.1.smithi045.stdout:May 03 19:03:34 smithi045 podman[68584]: 2024-05-03 19:03:34.469319269 +0000 UTC m=+0.126684550 container create d4f75d6aff95e338b2842d821f5444cac13bd977ae6082b12067367d210c7884 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-1-deactivate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240430, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.schema-version=1.0) 2024-05-03T19:03:35.152 INFO:journalctl@ceph.osd.1.smithi045.stdout:May 03 19:03:34 smithi045 podman[68584]: 2024-05-03 19:03:34.847255809 +0000 UTC m=+0.504621095 container init d4f75d6aff95e338b2842d821f5444cac13bd977ae6082b12067367d210c7884 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-1-deactivate, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240430, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-05-03T19:03:35.152 INFO:journalctl@ceph.osd.1.smithi045.stdout:May 03 19:03:34 smithi045 podman[68584]: 2024-05-03 19:03:34.850763547 +0000 UTC m=+0.508128823 container start d4f75d6aff95e338b2842d821f5444cac13bd977ae6082b12067367d210c7884 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-1-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240430, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-05-03T19:03:35.152 INFO:journalctl@ceph.osd.1.smithi045.stdout:May 03 19:03:34 smithi045 podman[68584]: 2024-05-03 19:03:34.93887381 +0000 UTC m=+0.596239096 container attach d4f75d6aff95e338b2842d821f5444cac13bd977ae6082b12067367d210c7884 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-1-deactivate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.build-date=20240430, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-05-03T19:03:35.435 INFO:journalctl@ceph.osd.1.smithi045.stdout:May 03 19:03:35 smithi045 podman[68584]: 2024-05-03 19:03:35.151567855 +0000 UTC m=+0.808933147 container died d4f75d6aff95e338b2842d821f5444cac13bd977ae6082b12067367d210c7884 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-1-deactivate, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240430, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-05-03T19:03:35.723 DEBUG:teuthology.orchestra.run.smithi045:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f500455e-097b-11ef-bc97-c7b262605968@osd.1.service' 2024-05-03T19:03:35.764 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-03T19:03:35.764 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-05-03T19:03:35.764 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-05-03T19:03:35.765 DEBUG:teuthology.orchestra.run.smithi113:> sudo systemctl stop ceph-f500455e-097b-11ef-bc97-c7b262605968@osd.2 2024-05-03T19:03:36.051 INFO:journalctl@ceph.osd.2.smithi113.stdout:May 03 19:03:35 smithi113 systemd[1]: Stopping Ceph osd.2 for f500455e-097b-11ef-bc97-c7b262605968... 2024-05-03T19:03:36.551 INFO:journalctl@ceph.osd.2.smithi113.stdout:May 03 19:03:36 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-2[38708]: 2024-05-03T19:03:36.085+0000 7faf364c9700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-03T19:03:36.551 INFO:journalctl@ceph.osd.2.smithi113.stdout:May 03 19:03:36 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-2[38708]: 2024-05-03T19:03:36.085+0000 7faf364c9700 -1 osd.2 85 *** Got signal Terminated *** 2024-05-03T19:03:36.552 INFO:journalctl@ceph.osd.2.smithi113.stdout:May 03 19:03:36 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-2[38708]: 2024-05-03T19:03:36.085+0000 7faf364c9700 -1 osd.2 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-03T19:03:41.879 INFO:journalctl@ceph.osd.2.smithi113.stdout:May 03 19:03:41 smithi113 podman[70065]: 2024-05-03 19:03:41.574544717 +0000 UTC m=+5.615792541 container died 20f0cd67f620d75346f03795ee2acd7b6b9db650a1a47d3f73ca595015cb236f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-2, org.label-schema.build-date=20240430, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True) 2024-05-03T19:03:42.237 INFO:journalctl@ceph.osd.2.smithi113.stdout:May 03 19:03:41 smithi113 podman[70065]: 2024-05-03 19:03:41.968994311 +0000 UTC m=+6.010242125 container cleanup 20f0cd67f620d75346f03795ee2acd7b6b9db650a1a47d3f73ca595015cb236f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-2, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240430, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True) 2024-05-03T19:03:42.238 INFO:journalctl@ceph.osd.2.smithi113.stdout:May 03 19:03:41 smithi113 bash[70065]: ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-2 2024-05-03T19:03:42.551 INFO:journalctl@ceph.osd.2.smithi113.stdout:May 03 19:03:42 smithi113 podman[70083]: 2024-05-03 19:03:42.237065599 +0000 UTC m=+0.661283245 container remove 20f0cd67f620d75346f03795ee2acd7b6b9db650a1a47d3f73ca595015cb236f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240430, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5) 2024-05-03T19:03:42.858 INFO:journalctl@ceph.osd.2.smithi113.stdout:May 03 19:03:42 smithi113 podman[70174]: 2024-05-03 19:03:42.487625609 +0000 UTC m=+0.018042247 image pull ee4d846202496ba6096593e8e8becc0700a5eb7ebd95536031187f5f337d4012 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd 2024-05-03T19:03:42.858 INFO:journalctl@ceph.osd.2.smithi113.stdout:May 03 19:03:42 smithi113 podman[70174]: 2024-05-03 19:03:42.588132573 +0000 UTC m=+0.118549240 container create bd0beed80b90110160de179483cb95183965b697b36d2c451a2489eca0c80aaa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-2-deactivate, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240430, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-05-03T19:03:42.858 INFO:journalctl@ceph.osd.2.smithi113.stdout:May 03 19:03:42 smithi113 podman[70174]: 2024-05-03 19:03:42.85775759 +0000 UTC m=+0.388174221 container init bd0beed80b90110160de179483cb95183965b697b36d2c451a2489eca0c80aaa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-2-deactivate, org.label-schema.build-date=20240430, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-05-03T19:03:43.150 INFO:journalctl@ceph.osd.2.smithi113.stdout:May 03 19:03:42 smithi113 podman[70174]: 2024-05-03 19:03:42.861462773 +0000 UTC m=+0.391879401 container start bd0beed80b90110160de179483cb95183965b697b36d2c451a2489eca0c80aaa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-2-deactivate, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240430, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5) 2024-05-03T19:03:43.150 INFO:journalctl@ceph.osd.2.smithi113.stdout:May 03 19:03:42 smithi113 podman[70174]: 2024-05-03 19:03:42.95757946 +0000 UTC m=+0.487996086 container attach bd0beed80b90110160de179483cb95183965b697b36d2c451a2489eca0c80aaa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-2-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240430, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True) 2024-05-03T19:03:43.426 INFO:journalctl@ceph.osd.2.smithi113.stdout:May 03 19:03:43 smithi113 podman[70174]: 2024-05-03 19:03:43.149921418 +0000 UTC m=+0.680338045 container died bd0beed80b90110160de179483cb95183965b697b36d2c451a2489eca0c80aaa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240430, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-05-03T19:03:43.715 DEBUG:teuthology.orchestra.run.smithi113:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f500455e-097b-11ef-bc97-c7b262605968@osd.2.service' 2024-05-03T19:03:43.753 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-03T19:03:43.753 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-05-03T19:03:43.753 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-05-03T19:03:43.754 DEBUG:teuthology.orchestra.run.smithi032:> sudo systemctl stop ceph-f500455e-097b-11ef-bc97-c7b262605968@osd.3 2024-05-03T19:03:44.069 INFO:journalctl@ceph.osd.3.smithi032.stdout:May 03 19:03:43 smithi032 systemd[1]: Stopping Ceph osd.3 for f500455e-097b-11ef-bc97-c7b262605968... 2024-05-03T19:03:44.568 INFO:journalctl@ceph.osd.3.smithi032.stdout:May 03 19:03:44 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-3[49213]: 2024-05-03T19:03:44.072+0000 7f5856c0a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-03T19:03:44.569 INFO:journalctl@ceph.osd.3.smithi032.stdout:May 03 19:03:44 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-3[49213]: 2024-05-03T19:03:44.072+0000 7f5856c0a700 -1 osd.3 85 *** Got signal Terminated *** 2024-05-03T19:03:44.569 INFO:journalctl@ceph.osd.3.smithi032.stdout:May 03 19:03:44 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-3[49213]: 2024-05-03T19:03:44.072+0000 7f5856c0a700 -1 osd.3 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-03T19:03:47.068 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:03:46 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:03:46.792+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:23.927793+0000 front 2024-05-03T19:03:23.927723+0000 (oldest deadline 2024-05-03T19:03:46.227921+0000) 2024-05-03T19:03:47.569 INFO:journalctl@ceph.osd.3.smithi032.stdout:May 03 19:03:47 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-3[49213]: 2024-05-03T19:03:47.263+0000 7f584f1f6700 -1 osd.3 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:23.974451+0000 front 2024-05-03T19:03:23.974483+0000 (oldest deadline 2024-05-03T19:03:46.274651+0000) 2024-05-03T19:03:47.569 INFO:journalctl@ceph.osd.6.smithi032.stdout:May 03 19:03:47 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-6[55806]: 2024-05-03T19:03:47.364+0000 7f7d9b701700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:21.662275+0000 front 2024-05-03T19:03:21.662309+0000 (oldest deadline 2024-05-03T19:03:46.962364+0000) 2024-05-03T19:03:47.801 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:03:47 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:03:47.356+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:25.060144+0000 front 2024-05-03T19:03:25.060125+0000 (oldest deadline 2024-05-03T19:03:46.760321+0000) 2024-05-03T19:03:48.068 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:03:47 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:03:47.771+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:23.927793+0000 front 2024-05-03T19:03:23.927723+0000 (oldest deadline 2024-05-03T19:03:46.227921+0000) 2024-05-03T19:03:48.568 INFO:journalctl@ceph.osd.3.smithi032.stdout:May 03 19:03:48 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-3[49213]: 2024-05-03T19:03:48.222+0000 7f584f1f6700 -1 osd.3 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:23.974451+0000 front 2024-05-03T19:03:23.974483+0000 (oldest deadline 2024-05-03T19:03:46.274651+0000) 2024-05-03T19:03:48.569 INFO:journalctl@ceph.osd.6.smithi032.stdout:May 03 19:03:48 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-6[55806]: 2024-05-03T19:03:48.336+0000 7f7d9b701700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:21.662275+0000 front 2024-05-03T19:03:21.662309+0000 (oldest deadline 2024-05-03T19:03:46.962364+0000) 2024-05-03T19:03:48.801 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:03:48 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:03:48.379+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:25.060144+0000 front 2024-05-03T19:03:25.060125+0000 (oldest deadline 2024-05-03T19:03:46.760321+0000) 2024-05-03T19:03:49.069 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:03:48 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:03:48.792+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:23.927793+0000 front 2024-05-03T19:03:23.927723+0000 (oldest deadline 2024-05-03T19:03:46.227921+0000) 2024-05-03T19:03:49.748 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:03:49 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:03:49.428+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:25.050521+0000 front 2024-05-03T19:03:25.050536+0000 (oldest deadline 2024-05-03T19:03:49.150320+0000) 2024-05-03T19:03:49.801 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:03:49 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:03:49.412+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:25.060144+0000 front 2024-05-03T19:03:25.060125+0000 (oldest deadline 2024-05-03T19:03:46.760321+0000) 2024-05-03T19:03:49.819 INFO:journalctl@ceph.osd.6.smithi032.stdout:May 03 19:03:49 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-6[55806]: 2024-05-03T19:03:49.330+0000 7f7d9b701700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:21.662275+0000 front 2024-05-03T19:03:21.662309+0000 (oldest deadline 2024-05-03T19:03:46.962364+0000) 2024-05-03T19:03:49.819 INFO:journalctl@ceph.osd.3.smithi032.stdout:May 03 19:03:49 smithi032 podman[87755]: 2024-05-03 19:03:49.549777009 +0000 UTC m=+5.592977131 container died fdf2be52179dfdae909f712efbf02ac7ee2c21cce99931bf26ae6bee92e75cec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-3, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240430) 2024-05-03T19:03:50.301 INFO:journalctl@ceph.osd.8.smithi113.stdout:May 03 19:03:49 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-8[52667]: 2024-05-03T19:03:49.975+0000 7fb4a8e32700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:21.503361+0000 front 2024-05-03T19:03:21.503321+0000 (oldest deadline 2024-05-03T19:03:49.704026+0000) 2024-05-03T19:03:50.318 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:03:49 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:03:49.832+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:23.927793+0000 front 2024-05-03T19:03:23.927723+0000 (oldest deadline 2024-05-03T19:03:46.227921+0000) 2024-05-03T19:03:50.319 INFO:journalctl@ceph.osd.3.smithi032.stdout:May 03 19:03:50 smithi032 podman[87755]: 2024-05-03 19:03:50.021930082 +0000 UTC m=+6.065130198 container cleanup fdf2be52179dfdae909f712efbf02ac7ee2c21cce99931bf26ae6bee92e75cec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-3, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240430, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-05-03T19:03:50.319 INFO:journalctl@ceph.osd.3.smithi032.stdout:May 03 19:03:50 smithi032 bash[87755]: ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-3 2024-05-03T19:03:50.748 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:03:50 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:03:50.473+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:25.050521+0000 front 2024-05-03T19:03:25.050536+0000 (oldest deadline 2024-05-03T19:03:49.150320+0000) 2024-05-03T19:03:50.801 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:03:50 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:03:50.383+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:25.060144+0000 front 2024-05-03T19:03:25.060125+0000 (oldest deadline 2024-05-03T19:03:46.760321+0000) 2024-05-03T19:03:50.811 INFO:journalctl@ceph.osd.6.smithi032.stdout:May 03 19:03:50 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-6[55806]: 2024-05-03T19:03:50.317+0000 7f7d9b701700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:21.662275+0000 front 2024-05-03T19:03:21.662309+0000 (oldest deadline 2024-05-03T19:03:46.962364+0000) 2024-05-03T19:03:50.811 INFO:journalctl@ceph.osd.3.smithi032.stdout:May 03 19:03:50 smithi032 podman[87773]: 2024-05-03 19:03:50.406482406 +0000 UTC m=+0.852698730 container remove fdf2be52179dfdae909f712efbf02ac7ee2c21cce99931bf26ae6bee92e75cec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-3, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.build-date=20240430, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-03T19:03:51.068 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:03:50 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:03:50.857+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:23.927793+0000 front 2024-05-03T19:03:23.927723+0000 (oldest deadline 2024-05-03T19:03:46.227921+0000) 2024-05-03T19:03:51.069 INFO:journalctl@ceph.osd.3.smithi032.stdout:May 03 19:03:50 smithi032 podman[87859]: 2024-05-03 19:03:50.713820402 +0000 UTC m=+0.018345503 image pull ee4d846202496ba6096593e8e8becc0700a5eb7ebd95536031187f5f337d4012 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd 2024-05-03T19:03:51.069 INFO:journalctl@ceph.osd.3.smithi032.stdout:May 03 19:03:50 smithi032 podman[87859]: 2024-05-03 19:03:50.832440704 +0000 UTC m=+0.136965842 container create bdefebcd788c02396dde070c4a39dc4bee8babf4c4ac76a43cfb3d0552c99cd4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-3-deactivate, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240430, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-05-03T19:03:51.069 INFO:journalctl@ceph.osd.3.smithi032.stdout:May 03 19:03:51 smithi032 podman[87859]: 2024-05-03 19:03:51.051795573 +0000 UTC m=+0.356320669 container init bdefebcd788c02396dde070c4a39dc4bee8babf4c4ac76a43cfb3d0552c99cd4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-3-deactivate, org.label-schema.build-date=20240430, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-03T19:03:51.069 INFO:journalctl@ceph.osd.3.smithi032.stdout:May 03 19:03:51 smithi032 podman[87859]: 2024-05-03 19:03:51.055596554 +0000 UTC m=+0.360121649 container start bdefebcd788c02396dde070c4a39dc4bee8babf4c4ac76a43cfb3d0552c99cd4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-3-deactivate, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240430, ceph=True) 2024-05-03T19:03:51.302 INFO:journalctl@ceph.osd.5.smithi113.stdout:May 03 19:03:50 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-5[45660]: 2024-05-03T19:03:50.869+0000 7f0c622a5700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:24.587577+0000 front 2024-05-03T19:03:24.587593+0000 (oldest deadline 2024-05-03T19:03:50.487474+0000) 2024-05-03T19:03:51.302 INFO:journalctl@ceph.osd.8.smithi113.stdout:May 03 19:03:50 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-8[52667]: 2024-05-03T19:03:50.952+0000 7fb4a8e32700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:21.503361+0000 front 2024-05-03T19:03:21.503321+0000 (oldest deadline 2024-05-03T19:03:49.704026+0000) 2024-05-03T19:03:51.568 INFO:journalctl@ceph.osd.6.smithi032.stdout:May 03 19:03:51 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-6[55806]: 2024-05-03T19:03:51.316+0000 7f7d9b701700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:21.662275+0000 front 2024-05-03T19:03:21.662309+0000 (oldest deadline 2024-05-03T19:03:46.962364+0000) 2024-05-03T19:03:51.569 INFO:journalctl@ceph.osd.3.smithi032.stdout:May 03 19:03:51 smithi032 podman[87859]: 2024-05-03 19:03:51.151686611 +0000 UTC m=+0.456211705 container attach bdefebcd788c02396dde070c4a39dc4bee8babf4c4ac76a43cfb3d0552c99cd4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-3-deactivate, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240430, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True) 2024-05-03T19:03:51.569 INFO:journalctl@ceph.osd.3.smithi032.stdout:May 03 19:03:51 smithi032 podman[87859]: 2024-05-03 19:03:51.365695868 +0000 UTC m=+0.670220976 container died bdefebcd788c02396dde070c4a39dc4bee8babf4c4ac76a43cfb3d0552c99cd4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-3-deactivate, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240430, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-05-03T19:03:51.748 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:03:51 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:03:51.454+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:25.050521+0000 front 2024-05-03T19:03:25.050536+0000 (oldest deadline 2024-05-03T19:03:49.150320+0000) 2024-05-03T19:03:51.801 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:03:51 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:03:51.363+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:25.060144+0000 front 2024-05-03T19:03:25.060125+0000 (oldest deadline 2024-05-03T19:03:46.760321+0000) 2024-05-03T19:03:51.905 DEBUG:teuthology.orchestra.run.smithi032:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f500455e-097b-11ef-bc97-c7b262605968@osd.3.service' 2024-05-03T19:03:51.928 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:03:51 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:03:51.894+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:23.927793+0000 front 2024-05-03T19:03:23.927723+0000 (oldest deadline 2024-05-03T19:03:46.227921+0000) 2024-05-03T19:03:51.928 INFO:journalctl@ceph.osd.3.smithi032.stdout:May 03 19:03:51 smithi032 podman[87885]: 2024-05-03 19:03:51.879761917 +0000 UTC m=+0.504355994 container remove bdefebcd788c02396dde070c4a39dc4bee8babf4c4ac76a43cfb3d0552c99cd4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-3-deactivate, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240430) 2024-05-03T19:03:51.929 INFO:journalctl@ceph.osd.3.smithi032.stdout:May 03 19:03:51 smithi032 systemd[1]: ceph-f500455e-097b-11ef-bc97-c7b262605968@osd.3.service: Deactivated successfully. 2024-05-03T19:03:51.929 INFO:journalctl@ceph.osd.3.smithi032.stdout:May 03 19:03:51 smithi032 systemd[1]: Stopped Ceph osd.3 for f500455e-097b-11ef-bc97-c7b262605968. 2024-05-03T19:03:51.929 INFO:journalctl@ceph.osd.3.smithi032.stdout:May 03 19:03:51 smithi032 systemd[1]: ceph-f500455e-097b-11ef-bc97-c7b262605968@osd.3.service: Consumed 11.366s CPU time. 2024-05-03T19:03:51.944 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-03T19:03:51.945 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-05-03T19:03:51.945 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-05-03T19:03:51.945 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ceph-f500455e-097b-11ef-bc97-c7b262605968@osd.4 2024-05-03T19:03:52.248 INFO:journalctl@ceph.osd.4.smithi045.stdout:May 03 19:03:51 smithi045 systemd[1]: Stopping Ceph osd.4 for f500455e-097b-11ef-bc97-c7b262605968... 2024-05-03T19:03:52.301 INFO:journalctl@ceph.osd.5.smithi113.stdout:May 03 19:03:51 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-5[45660]: 2024-05-03T19:03:51.854+0000 7f0c622a5700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:24.587577+0000 front 2024-05-03T19:03:24.587593+0000 (oldest deadline 2024-05-03T19:03:50.487474+0000) 2024-05-03T19:03:52.302 INFO:journalctl@ceph.osd.8.smithi113.stdout:May 03 19:03:51 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-8[52667]: 2024-05-03T19:03:51.925+0000 7fb4a8e32700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:21.503361+0000 front 2024-05-03T19:03:21.503321+0000 (oldest deadline 2024-05-03T19:03:49.704026+0000) 2024-05-03T19:03:52.748 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:03:52 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:03:52.470+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:25.050521+0000 front 2024-05-03T19:03:25.050536+0000 (oldest deadline 2024-05-03T19:03:49.150320+0000) 2024-05-03T19:03:52.749 INFO:journalctl@ceph.osd.4.smithi045.stdout:May 03 19:03:52 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-4[44832]: 2024-05-03T19:03:52.252+0000 7fc83cbd0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-03T19:03:52.749 INFO:journalctl@ceph.osd.4.smithi045.stdout:May 03 19:03:52 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-4[44832]: 2024-05-03T19:03:52.252+0000 7fc83cbd0700 -1 osd.4 85 *** Got signal Terminated *** 2024-05-03T19:03:52.749 INFO:journalctl@ceph.osd.4.smithi045.stdout:May 03 19:03:52 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-4[44832]: 2024-05-03T19:03:52.252+0000 7fc83cbd0700 -1 osd.4 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-03T19:03:52.801 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:03:52 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:03:52.379+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:25.060144+0000 front 2024-05-03T19:03:25.060125+0000 (oldest deadline 2024-05-03T19:03:46.760321+0000) 2024-05-03T19:03:52.818 INFO:journalctl@ceph.osd.6.smithi032.stdout:May 03 19:03:52 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-6[55806]: 2024-05-03T19:03:52.321+0000 7f7d9b701700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:21.662275+0000 front 2024-05-03T19:03:21.662309+0000 (oldest deadline 2024-05-03T19:03:46.962364+0000) 2024-05-03T19:03:53.301 INFO:journalctl@ceph.osd.8.smithi113.stdout:May 03 19:03:52 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-8[52667]: 2024-05-03T19:03:52.934+0000 7fb4a8e32700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:21.503361+0000 front 2024-05-03T19:03:21.503321+0000 (oldest deadline 2024-05-03T19:03:49.704026+0000) 2024-05-03T19:03:53.302 INFO:journalctl@ceph.osd.5.smithi113.stdout:May 03 19:03:52 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-5[45660]: 2024-05-03T19:03:52.894+0000 7f0c622a5700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:24.587577+0000 front 2024-05-03T19:03:24.587593+0000 (oldest deadline 2024-05-03T19:03:50.487474+0000) 2024-05-03T19:03:53.318 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:03:52 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:03:52.943+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:23.927793+0000 front 2024-05-03T19:03:23.927723+0000 (oldest deadline 2024-05-03T19:03:46.227921+0000) 2024-05-03T19:03:53.801 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:03:53 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:03:53.407+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:25.060144+0000 front 2024-05-03T19:03:25.060125+0000 (oldest deadline 2024-05-03T19:03:46.760321+0000) 2024-05-03T19:03:53.818 INFO:journalctl@ceph.osd.6.smithi032.stdout:May 03 19:03:53 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-6[55806]: 2024-05-03T19:03:53.342+0000 7f7d9b701700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:21.662275+0000 front 2024-05-03T19:03:21.662309+0000 (oldest deadline 2024-05-03T19:03:46.962364+0000) 2024-05-03T19:03:53.998 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:03:53 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:03:53.501+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:25.050521+0000 front 2024-05-03T19:03:25.050536+0000 (oldest deadline 2024-05-03T19:03:49.150320+0000) 2024-05-03T19:03:54.301 INFO:journalctl@ceph.osd.5.smithi113.stdout:May 03 19:03:53 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-5[45660]: 2024-05-03T19:03:53.917+0000 7f0c622a5700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:24.587577+0000 front 2024-05-03T19:03:24.587593+0000 (oldest deadline 2024-05-03T19:03:50.487474+0000) 2024-05-03T19:03:54.302 INFO:journalctl@ceph.osd.8.smithi113.stdout:May 03 19:03:53 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-8[52667]: 2024-05-03T19:03:53.933+0000 7fb4a8e32700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:21.503361+0000 front 2024-05-03T19:03:21.503321+0000 (oldest deadline 2024-05-03T19:03:49.704026+0000) 2024-05-03T19:03:54.302 INFO:journalctl@ceph.osd.8.smithi113.stdout:May 03 19:03:53 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-8[52667]: 2024-05-03T19:03:53.933+0000 7fb4a8e32700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:30.805173+0000 front 2024-05-03T19:03:30.805157+0000 (oldest deadline 2024-05-03T19:03:53.605897+0000) 2024-05-03T19:03:54.318 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:03:53 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:03:53.925+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:23.927793+0000 front 2024-05-03T19:03:23.927723+0000 (oldest deadline 2024-05-03T19:03:46.227921+0000) 2024-05-03T19:03:54.794 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:03:54 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:03:54.535+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:25.050521+0000 front 2024-05-03T19:03:25.050536+0000 (oldest deadline 2024-05-03T19:03:49.150320+0000) 2024-05-03T19:03:54.801 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:03:54 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:03:54.394+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:25.060144+0000 front 2024-05-03T19:03:25.060125+0000 (oldest deadline 2024-05-03T19:03:46.760321+0000) 2024-05-03T19:03:54.818 INFO:journalctl@ceph.osd.6.smithi032.stdout:May 03 19:03:54 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-6[55806]: 2024-05-03T19:03:54.326+0000 7f7d9b701700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:21.662275+0000 front 2024-05-03T19:03:21.662309+0000 (oldest deadline 2024-05-03T19:03:46.962364+0000) 2024-05-03T19:03:55.248 INFO:journalctl@ceph.osd.7.smithi045.stdout:May 03 19:03:54 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-7[51710]: 2024-05-03T19:03:54.793+0000 7fe2c0194700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:32.077137+0000 front 2024-05-03T19:03:32.077303+0000 (oldest deadline 2024-05-03T19:03:53.777433+0000) 2024-05-03T19:03:55.283 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:03:54 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:03:54.942+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:23.927793+0000 front 2024-05-03T19:03:23.927723+0000 (oldest deadline 2024-05-03T19:03:46.227921+0000) 2024-05-03T19:03:55.283 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:03:54 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:03:54.942+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:29.729118+0000 front 2024-05-03T19:03:29.729178+0000 (oldest deadline 2024-05-03T19:03:54.429041+0000) 2024-05-03T19:03:55.301 INFO:journalctl@ceph.osd.5.smithi113.stdout:May 03 19:03:54 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-5[45660]: 2024-05-03T19:03:54.873+0000 7f0c622a5700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:24.587577+0000 front 2024-05-03T19:03:24.587593+0000 (oldest deadline 2024-05-03T19:03:50.487474+0000) 2024-05-03T19:03:55.302 INFO:journalctl@ceph.osd.8.smithi113.stdout:May 03 19:03:54 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-8[52667]: 2024-05-03T19:03:54.977+0000 7fb4a8e32700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:21.503361+0000 front 2024-05-03T19:03:21.503321+0000 (oldest deadline 2024-05-03T19:03:49.704026+0000) 2024-05-03T19:03:55.302 INFO:journalctl@ceph.osd.8.smithi113.stdout:May 03 19:03:54 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-8[52667]: 2024-05-03T19:03:54.977+0000 7fb4a8e32700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:30.805173+0000 front 2024-05-03T19:03:30.805157+0000 (oldest deadline 2024-05-03T19:03:53.605897+0000) 2024-05-03T19:03:55.568 INFO:journalctl@ceph.osd.6.smithi032.stdout:May 03 19:03:55 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-6[55806]: 2024-05-03T19:03:55.282+0000 7f7d9b701700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:21.662275+0000 front 2024-05-03T19:03:21.662309+0000 (oldest deadline 2024-05-03T19:03:46.962364+0000) 2024-05-03T19:03:55.801 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:03:55 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:03:55.388+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:25.060144+0000 front 2024-05-03T19:03:25.060125+0000 (oldest deadline 2024-05-03T19:03:46.760321+0000) 2024-05-03T19:03:55.948 INFO:journalctl@ceph.osd.7.smithi045.stdout:May 03 19:03:55 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-7[51710]: 2024-05-03T19:03:55.801+0000 7fe2c0194700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:32.077137+0000 front 2024-05-03T19:03:32.077303+0000 (oldest deadline 2024-05-03T19:03:53.777433+0000) 2024-05-03T19:03:55.948 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:03:55 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:03:55.576+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:25.050521+0000 front 2024-05-03T19:03:25.050536+0000 (oldest deadline 2024-05-03T19:03:49.150320+0000) 2024-05-03T19:03:55.948 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:03:55 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:03:55.576+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:29.150598+0000 front 2024-05-03T19:03:29.150627+0000 (oldest deadline 2024-05-03T19:03:55.050741+0000) 2024-05-03T19:03:56.248 INFO:journalctl@ceph.osd.4.smithi045.stdout:May 03 19:03:55 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-4[44832]: 2024-05-03T19:03:55.946+0000 7fc8391c4700 -1 osd.4 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:29.396786+0000 front 2024-05-03T19:03:29.396808+0000 (oldest deadline 2024-05-03T19:03:55.296937+0000) 2024-05-03T19:03:56.301 INFO:journalctl@ceph.osd.5.smithi113.stdout:May 03 19:03:55 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-5[45660]: 2024-05-03T19:03:55.828+0000 7f0c622a5700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:24.587577+0000 front 2024-05-03T19:03:24.587593+0000 (oldest deadline 2024-05-03T19:03:50.487474+0000) 2024-05-03T19:03:56.302 INFO:journalctl@ceph.osd.8.smithi113.stdout:May 03 19:03:55 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-8[52667]: 2024-05-03T19:03:55.986+0000 7fb4a8e32700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:21.503361+0000 front 2024-05-03T19:03:21.503321+0000 (oldest deadline 2024-05-03T19:03:49.704026+0000) 2024-05-03T19:03:56.302 INFO:journalctl@ceph.osd.8.smithi113.stdout:May 03 19:03:55 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-8[52667]: 2024-05-03T19:03:55.986+0000 7fb4a8e32700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:30.805173+0000 front 2024-05-03T19:03:30.805157+0000 (oldest deadline 2024-05-03T19:03:53.605897+0000) 2024-05-03T19:03:56.302 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:03:55 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:03:55.954+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:23.927793+0000 front 2024-05-03T19:03:23.927723+0000 (oldest deadline 2024-05-03T19:03:46.227921+0000) 2024-05-03T19:03:56.303 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:03:55 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:03:55.954+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:29.729118+0000 front 2024-05-03T19:03:29.729178+0000 (oldest deadline 2024-05-03T19:03:54.429041+0000) 2024-05-03T19:03:56.568 INFO:journalctl@ceph.osd.6.smithi032.stdout:May 03 19:03:56 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-6[55806]: 2024-05-03T19:03:56.301+0000 7f7d9b701700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:21.662275+0000 front 2024-05-03T19:03:21.662309+0000 (oldest deadline 2024-05-03T19:03:46.962364+0000) 2024-05-03T19:03:56.801 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:03:56 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:03:56.395+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:25.060144+0000 front 2024-05-03T19:03:25.060125+0000 (oldest deadline 2024-05-03T19:03:46.760321+0000) 2024-05-03T19:03:56.816 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:03:56 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:03:56.534+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:25.050521+0000 front 2024-05-03T19:03:25.050536+0000 (oldest deadline 2024-05-03T19:03:49.150320+0000) 2024-05-03T19:03:56.816 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:03:56 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:03:56.534+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:29.150598+0000 front 2024-05-03T19:03:29.150627+0000 (oldest deadline 2024-05-03T19:03:55.050741+0000) 2024-05-03T19:03:57.248 INFO:journalctl@ceph.osd.7.smithi045.stdout:May 03 19:03:56 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-7[51710]: 2024-05-03T19:03:56.814+0000 7fe2c0194700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:32.077137+0000 front 2024-05-03T19:03:32.077303+0000 (oldest deadline 2024-05-03T19:03:53.777433+0000) 2024-05-03T19:03:57.248 INFO:journalctl@ceph.osd.4.smithi045.stdout:May 03 19:03:56 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-4[44832]: 2024-05-03T19:03:56.901+0000 7fc8391c4700 -1 osd.4 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:29.396786+0000 front 2024-05-03T19:03:29.396808+0000 (oldest deadline 2024-05-03T19:03:55.296937+0000) 2024-05-03T19:03:57.262 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:03:56 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:03:56.929+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:23.927793+0000 front 2024-05-03T19:03:23.927723+0000 (oldest deadline 2024-05-03T19:03:46.227921+0000) 2024-05-03T19:03:57.262 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:03:56 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:03:56.929+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:29.729118+0000 front 2024-05-03T19:03:29.729178+0000 (oldest deadline 2024-05-03T19:03:54.429041+0000) 2024-05-03T19:03:57.301 INFO:journalctl@ceph.osd.5.smithi113.stdout:May 03 19:03:56 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-5[45660]: 2024-05-03T19:03:56.816+0000 7f0c622a5700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:24.587577+0000 front 2024-05-03T19:03:24.587593+0000 (oldest deadline 2024-05-03T19:03:50.487474+0000) 2024-05-03T19:03:57.301 INFO:journalctl@ceph.osd.8.smithi113.stdout:May 03 19:03:56 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-8[52667]: 2024-05-03T19:03:56.985+0000 7fb4a8e32700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:21.503361+0000 front 2024-05-03T19:03:21.503321+0000 (oldest deadline 2024-05-03T19:03:49.704026+0000) 2024-05-03T19:03:57.301 INFO:journalctl@ceph.osd.8.smithi113.stdout:May 03 19:03:56 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-8[52667]: 2024-05-03T19:03:56.985+0000 7fb4a8e32700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:30.805173+0000 front 2024-05-03T19:03:30.805157+0000 (oldest deadline 2024-05-03T19:03:53.605897+0000) 2024-05-03T19:03:57.568 INFO:journalctl@ceph.osd.6.smithi032.stdout:May 03 19:03:57 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-6[55806]: 2024-05-03T19:03:57.260+0000 7f7d9b701700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:21.662275+0000 front 2024-05-03T19:03:21.662309+0000 (oldest deadline 2024-05-03T19:03:46.962364+0000) 2024-05-03T19:03:57.801 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:03:57 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:03:57.365+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:25.060144+0000 front 2024-05-03T19:03:25.060125+0000 (oldest deadline 2024-05-03T19:03:46.760321+0000) 2024-05-03T19:03:57.801 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:03:57 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:03:57.365+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:31.362284+0000 front 2024-05-03T19:03:31.362156+0000 (oldest deadline 2024-05-03T19:03:56.662340+0000) 2024-05-03T19:03:57.857 INFO:journalctl@ceph.osd.4.smithi045.stdout:May 03 19:03:57 smithi045 podman[68691]: 2024-05-03 19:03:57.749034383 +0000 UTC m=+5.605600264 container died d1a649f525495d9a213500ad60130bd9c3f56beef859fb4444e706a2a079c9ab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-4, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240430, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-05-03T19:03:57.858 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:03:57 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:03:57.515+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:25.050521+0000 front 2024-05-03T19:03:25.050536+0000 (oldest deadline 2024-05-03T19:03:49.150320+0000) 2024-05-03T19:03:57.858 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:03:57 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:03:57.515+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:29.150598+0000 front 2024-05-03T19:03:29.150627+0000 (oldest deadline 2024-05-03T19:03:55.050741+0000) 2024-05-03T19:03:58.145 INFO:journalctl@ceph.osd.7.smithi045.stdout:May 03 19:03:57 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-7[51710]: 2024-05-03T19:03:57.855+0000 7fe2c0194700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:32.077137+0000 front 2024-05-03T19:03:32.077303+0000 (oldest deadline 2024-05-03T19:03:53.777433+0000) 2024-05-03T19:03:58.231 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:03:57 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:03:57.947+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:23.927793+0000 front 2024-05-03T19:03:23.927723+0000 (oldest deadline 2024-05-03T19:03:46.227921+0000) 2024-05-03T19:03:58.231 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:03:57 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:03:57.947+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:29.729118+0000 front 2024-05-03T19:03:29.729178+0000 (oldest deadline 2024-05-03T19:03:54.429041+0000) 2024-05-03T19:03:58.301 INFO:journalctl@ceph.osd.8.smithi113.stdout:May 03 19:03:57 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-8[52667]: 2024-05-03T19:03:57.983+0000 7fb4a8e32700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:21.503361+0000 front 2024-05-03T19:03:21.503321+0000 (oldest deadline 2024-05-03T19:03:49.704026+0000) 2024-05-03T19:03:58.301 INFO:journalctl@ceph.osd.8.smithi113.stdout:May 03 19:03:57 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-8[52667]: 2024-05-03T19:03:57.983+0000 7fb4a8e32700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:30.805173+0000 front 2024-05-03T19:03:30.805157+0000 (oldest deadline 2024-05-03T19:03:53.605897+0000) 2024-05-03T19:03:58.302 INFO:journalctl@ceph.osd.5.smithi113.stdout:May 03 19:03:57 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-5[45660]: 2024-05-03T19:03:57.861+0000 7f0c622a5700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:24.587577+0000 front 2024-05-03T19:03:24.587593+0000 (oldest deadline 2024-05-03T19:03:50.487474+0000) 2024-05-03T19:03:58.496 INFO:journalctl@ceph.osd.4.smithi045.stdout:May 03 19:03:58 smithi045 podman[68691]: 2024-05-03 19:03:58.219204951 +0000 UTC m=+6.075770819 container cleanup d1a649f525495d9a213500ad60130bd9c3f56beef859fb4444e706a2a079c9ab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-4, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240430, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-03T19:03:58.496 INFO:journalctl@ceph.osd.4.smithi045.stdout:May 03 19:03:58 smithi045 bash[68691]: ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-4 2024-05-03T19:03:58.570 INFO:journalctl@ceph.osd.6.smithi032.stdout:May 03 19:03:58 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-6[55806]: 2024-05-03T19:03:58.229+0000 7f7d9b701700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:21.662275+0000 front 2024-05-03T19:03:21.662309+0000 (oldest deadline 2024-05-03T19:03:46.962364+0000) 2024-05-03T19:03:58.570 INFO:journalctl@ceph.osd.6.smithi032.stdout:May 03 19:03:58 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-6[55806]: 2024-05-03T19:03:58.229+0000 7f7d9b701700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:32.863448+0000 front 2024-05-03T19:03:32.863665+0000 (oldest deadline 2024-05-03T19:03:57.563487+0000) 2024-05-03T19:03:58.748 INFO:journalctl@ceph.osd.4.smithi045.stdout:May 03 19:03:58 smithi045 podman[68717]: 2024-05-03 19:03:58.495673116 +0000 UTC m=+0.739586072 container remove d1a649f525495d9a213500ad60130bd9c3f56beef859fb4444e706a2a079c9ab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-4, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240430) 2024-05-03T19:03:58.748 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:03:58 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:03:58.499+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:25.050521+0000 front 2024-05-03T19:03:25.050536+0000 (oldest deadline 2024-05-03T19:03:49.150320+0000) 2024-05-03T19:03:58.749 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:03:58 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:03:58.499+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:29.150598+0000 front 2024-05-03T19:03:29.150627+0000 (oldest deadline 2024-05-03T19:03:55.050741+0000) 2024-05-03T19:03:58.801 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:03:58 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:03:58.332+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:25.060144+0000 front 2024-05-03T19:03:25.060125+0000 (oldest deadline 2024-05-03T19:03:46.760321+0000) 2024-05-03T19:03:58.801 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:03:58 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:03:58.332+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:31.362284+0000 front 2024-05-03T19:03:31.362156+0000 (oldest deadline 2024-05-03T19:03:56.662340+0000) 2024-05-03T19:03:59.248 INFO:journalctl@ceph.osd.7.smithi045.stdout:May 03 19:03:58 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-7[51710]: 2024-05-03T19:03:58.881+0000 7fe2c0194700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:32.077137+0000 front 2024-05-03T19:03:32.077303+0000 (oldest deadline 2024-05-03T19:03:53.777433+0000) 2024-05-03T19:03:59.248 INFO:journalctl@ceph.osd.4.smithi045.stdout:May 03 19:03:58 smithi045 podman[68811]: 2024-05-03 19:03:58.754205967 +0000 UTC m=+0.018579226 image pull ee4d846202496ba6096593e8e8becc0700a5eb7ebd95536031187f5f337d4012 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd 2024-05-03T19:03:59.249 INFO:journalctl@ceph.osd.4.smithi045.stdout:May 03 19:03:58 smithi045 podman[68811]: 2024-05-03 19:03:58.871832615 +0000 UTC m=+0.136205901 container create a2c178574772648adb5c599740c72da75f8d791e4cd6aaa592180886cb1c16b0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-4-deactivate, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240430, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-05-03T19:03:59.273 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:03:58 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:03:58.965+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:23.927793+0000 front 2024-05-03T19:03:23.927723+0000 (oldest deadline 2024-05-03T19:03:46.227921+0000) 2024-05-03T19:03:59.274 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:03:58 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:03:58.965+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:29.729118+0000 front 2024-05-03T19:03:29.729178+0000 (oldest deadline 2024-05-03T19:03:54.429041+0000) 2024-05-03T19:03:59.294 INFO:journalctl@ceph.osd.5.smithi113.stdout:May 03 19:03:58 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-5[45660]: 2024-05-03T19:03:58.824+0000 7f0c622a5700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:24.587577+0000 front 2024-05-03T19:03:24.587593+0000 (oldest deadline 2024-05-03T19:03:50.487474+0000) 2024-05-03T19:03:59.294 INFO:journalctl@ceph.osd.5.smithi113.stdout:May 03 19:03:58 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-5[45660]: 2024-05-03T19:03:58.824+0000 7f0c622a5700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:32.788316+0000 front 2024-05-03T19:03:32.788339+0000 (oldest deadline 2024-05-03T19:03:58.088537+0000) 2024-05-03T19:03:59.294 INFO:journalctl@ceph.osd.8.smithi113.stdout:May 03 19:03:58 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-8[52667]: 2024-05-03T19:03:58.954+0000 7fb4a8e32700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:21.503361+0000 front 2024-05-03T19:03:21.503321+0000 (oldest deadline 2024-05-03T19:03:49.704026+0000) 2024-05-03T19:03:59.294 INFO:journalctl@ceph.osd.8.smithi113.stdout:May 03 19:03:58 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-8[52667]: 2024-05-03T19:03:58.954+0000 7fb4a8e32700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:30.805173+0000 front 2024-05-03T19:03:30.805157+0000 (oldest deadline 2024-05-03T19:03:53.605897+0000) 2024-05-03T19:03:59.544 INFO:journalctl@ceph.osd.4.smithi045.stdout:May 03 19:03:59 smithi045 podman[68811]: 2024-05-03 19:03:59.270014145 +0000 UTC m=+0.534387370 container init a2c178574772648adb5c599740c72da75f8d791e4cd6aaa592180886cb1c16b0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-4-deactivate, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240430, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-05-03T19:03:59.544 INFO:journalctl@ceph.osd.4.smithi045.stdout:May 03 19:03:59 smithi045 podman[68811]: 2024-05-03 19:03:59.273349676 +0000 UTC m=+0.537722901 container start a2c178574772648adb5c599740c72da75f8d791e4cd6aaa592180886cb1c16b0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-4-deactivate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240430, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-05-03T19:03:59.544 INFO:journalctl@ceph.osd.4.smithi045.stdout:May 03 19:03:59 smithi045 podman[68811]: 2024-05-03 19:03:59.357446965 +0000 UTC m=+0.621820195 container attach a2c178574772648adb5c599740c72da75f8d791e4cd6aaa592180886cb1c16b0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-4-deactivate, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240430, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0) 2024-05-03T19:03:59.551 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:03:59 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:03:59.292+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:25.060144+0000 front 2024-05-03T19:03:25.060125+0000 (oldest deadline 2024-05-03T19:03:46.760321+0000) 2024-05-03T19:03:59.551 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:03:59 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:03:59.292+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:31.362284+0000 front 2024-05-03T19:03:31.362156+0000 (oldest deadline 2024-05-03T19:03:56.662340+0000) 2024-05-03T19:03:59.568 INFO:journalctl@ceph.osd.6.smithi032.stdout:May 03 19:03:59 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-6[55806]: 2024-05-03T19:03:59.271+0000 7f7d9b701700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:21.662275+0000 front 2024-05-03T19:03:21.662309+0000 (oldest deadline 2024-05-03T19:03:46.962364+0000) 2024-05-03T19:03:59.568 INFO:journalctl@ceph.osd.6.smithi032.stdout:May 03 19:03:59 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-6[55806]: 2024-05-03T19:03:59.271+0000 7f7d9b701700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:32.863448+0000 front 2024-05-03T19:03:32.863665+0000 (oldest deadline 2024-05-03T19:03:57.563487+0000) 2024-05-03T19:03:59.862 INFO:journalctl@ceph.osd.4.smithi045.stdout:May 03 19:03:59 smithi045 podman[68811]: 2024-05-03 19:03:59.581010323 +0000 UTC m=+0.845383567 container died a2c178574772648adb5c599740c72da75f8d791e4cd6aaa592180886cb1c16b0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-4-deactivate, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240430) 2024-05-03T19:03:59.863 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:03:59 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:03:59.541+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:25.050521+0000 front 2024-05-03T19:03:25.050536+0000 (oldest deadline 2024-05-03T19:03:49.150320+0000) 2024-05-03T19:03:59.863 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:03:59 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:03:59.541+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:29.150598+0000 front 2024-05-03T19:03:29.150627+0000 (oldest deadline 2024-05-03T19:03:55.050741+0000) 2024-05-03T19:04:00.128 INFO:journalctl@ceph.osd.7.smithi045.stdout:May 03 19:03:59 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-7[51710]: 2024-05-03T19:03:59.861+0000 7fe2c0194700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:32.077137+0000 front 2024-05-03T19:03:32.077303+0000 (oldest deadline 2024-05-03T19:03:53.777433+0000) 2024-05-03T19:04:00.154 DEBUG:teuthology.orchestra.run.smithi045:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f500455e-097b-11ef-bc97-c7b262605968@osd.4.service' 2024-05-03T19:04:00.193 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-03T19:04:00.193 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-05-03T19:04:00.193 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-05-03T19:04:00.193 DEBUG:teuthology.orchestra.run.smithi113:> sudo systemctl stop ceph-f500455e-097b-11ef-bc97-c7b262605968@osd.5 2024-05-03T19:04:00.219 INFO:journalctl@ceph.osd.5.smithi113.stdout:May 03 19:03:59 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-5[45660]: 2024-05-03T19:03:59.832+0000 7f0c622a5700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:24.587577+0000 front 2024-05-03T19:03:24.587593+0000 (oldest deadline 2024-05-03T19:03:50.487474+0000) 2024-05-03T19:04:00.220 INFO:journalctl@ceph.osd.5.smithi113.stdout:May 03 19:03:59 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-5[45660]: 2024-05-03T19:03:59.832+0000 7f0c622a5700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:32.788316+0000 front 2024-05-03T19:03:32.788339+0000 (oldest deadline 2024-05-03T19:03:58.088537+0000) 2024-05-03T19:04:00.220 INFO:journalctl@ceph.osd.8.smithi113.stdout:May 03 19:03:59 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-8[52667]: 2024-05-03T19:03:59.954+0000 7fb4a8e32700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:21.503361+0000 front 2024-05-03T19:03:21.503321+0000 (oldest deadline 2024-05-03T19:03:49.704026+0000) 2024-05-03T19:04:00.220 INFO:journalctl@ceph.osd.8.smithi113.stdout:May 03 19:03:59 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-8[52667]: 2024-05-03T19:03:59.954+0000 7fb4a8e32700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:30.805173+0000 front 2024-05-03T19:03:30.805157+0000 (oldest deadline 2024-05-03T19:03:53.605897+0000) 2024-05-03T19:04:00.282 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:00 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:04:00.000+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:23.927793+0000 front 2024-05-03T19:03:23.927723+0000 (oldest deadline 2024-05-03T19:03:46.227921+0000) 2024-05-03T19:04:00.282 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:00 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:04:00.000+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:29.729118+0000 front 2024-05-03T19:03:29.729178+0000 (oldest deadline 2024-05-03T19:03:54.429041+0000) 2024-05-03T19:04:00.498 INFO:journalctl@ceph.osd.5.smithi113.stdout:May 03 19:04:00 smithi113 systemd[1]: Stopping Ceph osd.5 for f500455e-097b-11ef-bc97-c7b262605968... 2024-05-03T19:04:00.498 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:00 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:00.292+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:25.060144+0000 front 2024-05-03T19:03:25.060125+0000 (oldest deadline 2024-05-03T19:03:46.760321+0000) 2024-05-03T19:04:00.498 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:00 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:00.292+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:31.362284+0000 front 2024-05-03T19:03:31.362156+0000 (oldest deadline 2024-05-03T19:03:56.662340+0000) 2024-05-03T19:04:00.568 INFO:journalctl@ceph.osd.6.smithi032.stdout:May 03 19:04:00 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-6[55806]: 2024-05-03T19:04:00.281+0000 7f7d9b701700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:21.662275+0000 front 2024-05-03T19:03:21.662309+0000 (oldest deadline 2024-05-03T19:03:46.962364+0000) 2024-05-03T19:04:00.569 INFO:journalctl@ceph.osd.6.smithi032.stdout:May 03 19:04:00 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-6[55806]: 2024-05-03T19:04:00.281+0000 7f7d9b701700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:32.863448+0000 front 2024-05-03T19:03:32.863665+0000 (oldest deadline 2024-05-03T19:03:57.563487+0000) 2024-05-03T19:04:00.801 INFO:journalctl@ceph.osd.5.smithi113.stdout:May 03 19:04:00 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-5[45660]: 2024-05-03T19:04:00.495+0000 7f0c69cb9700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-03T19:04:00.802 INFO:journalctl@ceph.osd.5.smithi113.stdout:May 03 19:04:00 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-5[45660]: 2024-05-03T19:04:00.495+0000 7f0c69cb9700 -1 osd.5 85 *** Got signal Terminated *** 2024-05-03T19:04:00.802 INFO:journalctl@ceph.osd.5.smithi113.stdout:May 03 19:04:00 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-5[45660]: 2024-05-03T19:04:00.495+0000 7f0c69cb9700 -1 osd.5 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-03T19:04:00.817 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:00 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:00.498+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:25.050521+0000 front 2024-05-03T19:03:25.050536+0000 (oldest deadline 2024-05-03T19:03:49.150320+0000) 2024-05-03T19:04:00.817 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:00 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:00.498+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:29.150598+0000 front 2024-05-03T19:03:29.150627+0000 (oldest deadline 2024-05-03T19:03:55.050741+0000) 2024-05-03T19:04:01.248 INFO:journalctl@ceph.osd.7.smithi045.stdout:May 03 19:04:00 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-7[51710]: 2024-05-03T19:04:00.812+0000 7fe2c0194700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:32.077137+0000 front 2024-05-03T19:03:32.077303+0000 (oldest deadline 2024-05-03T19:03:53.777433+0000) 2024-05-03T19:04:01.263 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:00 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:04:00.982+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:23.927793+0000 front 2024-05-03T19:03:23.927723+0000 (oldest deadline 2024-05-03T19:03:46.227921+0000) 2024-05-03T19:04:01.263 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:00 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:04:00.982+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:29.729118+0000 front 2024-05-03T19:03:29.729178+0000 (oldest deadline 2024-05-03T19:03:54.429041+0000) 2024-05-03T19:04:01.276 INFO:journalctl@ceph.osd.5.smithi113.stdout:May 03 19:04:00 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-5[45660]: 2024-05-03T19:04:00.869+0000 7f0c622a5700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:24.587577+0000 front 2024-05-03T19:03:24.587593+0000 (oldest deadline 2024-05-03T19:03:50.487474+0000) 2024-05-03T19:04:01.277 INFO:journalctl@ceph.osd.5.smithi113.stdout:May 03 19:04:00 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-5[45660]: 2024-05-03T19:04:00.869+0000 7f0c622a5700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:32.788316+0000 front 2024-05-03T19:03:32.788339+0000 (oldest deadline 2024-05-03T19:03:58.088537+0000) 2024-05-03T19:04:01.277 INFO:journalctl@ceph.osd.8.smithi113.stdout:May 03 19:04:00 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-8[52667]: 2024-05-03T19:04:00.968+0000 7fb4a8e32700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:21.503361+0000 front 2024-05-03T19:03:21.503321+0000 (oldest deadline 2024-05-03T19:03:49.704026+0000) 2024-05-03T19:04:01.277 INFO:journalctl@ceph.osd.8.smithi113.stdout:May 03 19:04:00 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-8[52667]: 2024-05-03T19:04:00.968+0000 7fb4a8e32700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:30.805173+0000 front 2024-05-03T19:03:30.805157+0000 (oldest deadline 2024-05-03T19:03:53.605897+0000) 2024-05-03T19:04:01.551 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:01 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:01.275+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:25.060144+0000 front 2024-05-03T19:03:25.060125+0000 (oldest deadline 2024-05-03T19:03:46.760321+0000) 2024-05-03T19:04:01.551 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:01 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:01.275+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:31.362284+0000 front 2024-05-03T19:03:31.362156+0000 (oldest deadline 2024-05-03T19:03:56.662340+0000) 2024-05-03T19:04:01.568 INFO:journalctl@ceph.osd.6.smithi032.stdout:May 03 19:04:01 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-6[55806]: 2024-05-03T19:04:01.262+0000 7f7d9b701700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:21.662275+0000 front 2024-05-03T19:03:21.662309+0000 (oldest deadline 2024-05-03T19:03:46.962364+0000) 2024-05-03T19:04:01.568 INFO:journalctl@ceph.osd.6.smithi032.stdout:May 03 19:04:01 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-6[55806]: 2024-05-03T19:04:01.262+0000 7f7d9b701700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:32.863448+0000 front 2024-05-03T19:03:32.863665+0000 (oldest deadline 2024-05-03T19:03:57.563487+0000) 2024-05-03T19:04:01.748 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:01 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:01.489+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:25.050521+0000 front 2024-05-03T19:03:25.050536+0000 (oldest deadline 2024-05-03T19:03:49.150320+0000) 2024-05-03T19:04:01.748 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:01 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:01.489+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:29.150598+0000 front 2024-05-03T19:03:29.150627+0000 (oldest deadline 2024-05-03T19:03:55.050741+0000) 2024-05-03T19:04:02.248 INFO:journalctl@ceph.osd.7.smithi045.stdout:May 03 19:04:01 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-7[51710]: 2024-05-03T19:04:01.791+0000 7fe2c0194700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:32.077137+0000 front 2024-05-03T19:03:32.077303+0000 (oldest deadline 2024-05-03T19:03:53.777433+0000) 2024-05-03T19:04:02.296 INFO:journalctl@ceph.osd.5.smithi113.stdout:May 03 19:04:01 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-5[45660]: 2024-05-03T19:04:01.870+0000 7f0c622a5700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:24.587577+0000 front 2024-05-03T19:03:24.587593+0000 (oldest deadline 2024-05-03T19:03:50.487474+0000) 2024-05-03T19:04:02.296 INFO:journalctl@ceph.osd.5.smithi113.stdout:May 03 19:04:01 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-5[45660]: 2024-05-03T19:04:01.870+0000 7f0c622a5700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:32.788316+0000 front 2024-05-03T19:03:32.788339+0000 (oldest deadline 2024-05-03T19:03:58.088537+0000) 2024-05-03T19:04:02.297 INFO:journalctl@ceph.osd.8.smithi113.stdout:May 03 19:04:01 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-8[52667]: 2024-05-03T19:04:01.945+0000 7fb4a8e32700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:21.503361+0000 front 2024-05-03T19:03:21.503321+0000 (oldest deadline 2024-05-03T19:03:49.704026+0000) 2024-05-03T19:04:02.297 INFO:journalctl@ceph.osd.8.smithi113.stdout:May 03 19:04:01 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-8[52667]: 2024-05-03T19:04:01.945+0000 7fb4a8e32700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:30.805173+0000 front 2024-05-03T19:03:30.805157+0000 (oldest deadline 2024-05-03T19:03:53.605897+0000) 2024-05-03T19:04:02.311 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:01 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:04:01.986+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:23.927793+0000 front 2024-05-03T19:03:23.927723+0000 (oldest deadline 2024-05-03T19:03:46.227921+0000) 2024-05-03T19:04:02.312 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:01 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:04:01.986+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:29.729118+0000 front 2024-05-03T19:03:29.729178+0000 (oldest deadline 2024-05-03T19:03:54.429041+0000) 2024-05-03T19:04:02.551 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:02 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:02.294+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:25.060144+0000 front 2024-05-03T19:03:25.060125+0000 (oldest deadline 2024-05-03T19:03:46.760321+0000) 2024-05-03T19:04:02.552 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:02 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:02.294+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:31.362284+0000 front 2024-05-03T19:03:31.362156+0000 (oldest deadline 2024-05-03T19:03:56.662340+0000) 2024-05-03T19:04:02.568 INFO:journalctl@ceph.osd.6.smithi032.stdout:May 03 19:04:02 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-6[55806]: 2024-05-03T19:04:02.310+0000 7f7d9b701700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:21.662275+0000 front 2024-05-03T19:03:21.662309+0000 (oldest deadline 2024-05-03T19:03:46.962364+0000) 2024-05-03T19:04:02.568 INFO:journalctl@ceph.osd.6.smithi032.stdout:May 03 19:04:02 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-6[55806]: 2024-05-03T19:04:02.310+0000 7f7d9b701700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:32.863448+0000 front 2024-05-03T19:03:32.863665+0000 (oldest deadline 2024-05-03T19:03:57.563487+0000) 2024-05-03T19:04:02.748 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:02 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:02.464+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:25.050521+0000 front 2024-05-03T19:03:25.050536+0000 (oldest deadline 2024-05-03T19:03:49.150320+0000) 2024-05-03T19:04:02.748 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:02 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:02.464+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:29.150598+0000 front 2024-05-03T19:03:29.150627+0000 (oldest deadline 2024-05-03T19:03:55.050741+0000) 2024-05-03T19:04:03.248 INFO:journalctl@ceph.osd.7.smithi045.stdout:May 03 19:04:02 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-7[51710]: 2024-05-03T19:04:02.802+0000 7fe2c0194700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:32.077137+0000 front 2024-05-03T19:03:32.077303+0000 (oldest deadline 2024-05-03T19:03:53.777433+0000) 2024-05-03T19:04:03.250 INFO:journalctl@ceph.osd.5.smithi113.stdout:May 03 19:04:02 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-5[45660]: 2024-05-03T19:04:02.838+0000 7f0c622a5700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:24.587577+0000 front 2024-05-03T19:03:24.587593+0000 (oldest deadline 2024-05-03T19:03:50.487474+0000) 2024-05-03T19:04:03.250 INFO:journalctl@ceph.osd.5.smithi113.stdout:May 03 19:04:02 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-5[45660]: 2024-05-03T19:04:02.838+0000 7f0c622a5700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:32.788316+0000 front 2024-05-03T19:03:32.788339+0000 (oldest deadline 2024-05-03T19:03:58.088537+0000) 2024-05-03T19:04:03.251 INFO:journalctl@ceph.osd.5.smithi113.stdout:May 03 19:04:02 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-5[45660]: 2024-05-03T19:04:02.838+0000 7f0c622a5700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:38.589504+0000 front 2024-05-03T19:03:38.589555+0000 (oldest deadline 2024-05-03T19:04:02.089599+0000) 2024-05-03T19:04:03.251 INFO:journalctl@ceph.osd.8.smithi113.stdout:May 03 19:04:02 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-8[52667]: 2024-05-03T19:04:02.959+0000 7fb4a8e32700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:21.503361+0000 front 2024-05-03T19:03:21.503321+0000 (oldest deadline 2024-05-03T19:03:49.704026+0000) 2024-05-03T19:04:03.251 INFO:journalctl@ceph.osd.8.smithi113.stdout:May 03 19:04:02 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-8[52667]: 2024-05-03T19:04:02.959+0000 7fb4a8e32700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:30.805173+0000 front 2024-05-03T19:03:30.805157+0000 (oldest deadline 2024-05-03T19:03:53.605897+0000) 2024-05-03T19:04:03.251 INFO:journalctl@ceph.osd.8.smithi113.stdout:May 03 19:04:02 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-8[52667]: 2024-05-03T19:04:02.959+0000 7fb4a8e32700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:37.106650+0000 front 2024-05-03T19:03:37.106721+0000 (oldest deadline 2024-05-03T19:04:02.406804+0000) 2024-05-03T19:04:03.318 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:03 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:04:03.013+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:23.927793+0000 front 2024-05-03T19:03:23.927723+0000 (oldest deadline 2024-05-03T19:03:46.227921+0000) 2024-05-03T19:04:03.319 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:03 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:04:03.013+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:29.729118+0000 front 2024-05-03T19:03:29.729178+0000 (oldest deadline 2024-05-03T19:03:54.429041+0000) 2024-05-03T19:04:03.551 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:03 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:03.248+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:25.060144+0000 front 2024-05-03T19:03:25.060125+0000 (oldest deadline 2024-05-03T19:03:46.760321+0000) 2024-05-03T19:04:03.551 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:03 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:03.248+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:31.362284+0000 front 2024-05-03T19:03:31.362156+0000 (oldest deadline 2024-05-03T19:03:56.662340+0000) 2024-05-03T19:04:03.779 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:03 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:03.498+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:25.050521+0000 front 2024-05-03T19:03:25.050536+0000 (oldest deadline 2024-05-03T19:03:49.150320+0000) 2024-05-03T19:04:03.779 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:03 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:03.498+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:29.150598+0000 front 2024-05-03T19:03:29.150627+0000 (oldest deadline 2024-05-03T19:03:55.050741+0000) 2024-05-03T19:04:03.780 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:03 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:03.498+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:37.951720+0000 front 2024-05-03T19:03:37.951864+0000 (oldest deadline 2024-05-03T19:04:02.651644+0000) 2024-05-03T19:04:03.818 INFO:journalctl@ceph.osd.6.smithi032.stdout:May 03 19:04:03 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-6[55806]: 2024-05-03T19:04:03.334+0000 7f7d9b701700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:21.662275+0000 front 2024-05-03T19:03:21.662309+0000 (oldest deadline 2024-05-03T19:03:46.962364+0000) 2024-05-03T19:04:03.818 INFO:journalctl@ceph.osd.6.smithi032.stdout:May 03 19:04:03 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-6[55806]: 2024-05-03T19:04:03.334+0000 7f7d9b701700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:32.863448+0000 front 2024-05-03T19:03:32.863665+0000 (oldest deadline 2024-05-03T19:03:57.563487+0000) 2024-05-03T19:04:04.051 INFO:journalctl@ceph.osd.5.smithi113.stdout:May 03 19:04:03 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-5[45660]: 2024-05-03T19:04:03.790+0000 7f0c622a5700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:24.587577+0000 front 2024-05-03T19:03:24.587593+0000 (oldest deadline 2024-05-03T19:03:50.487474+0000) 2024-05-03T19:04:04.052 INFO:journalctl@ceph.osd.5.smithi113.stdout:May 03 19:04:03 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-5[45660]: 2024-05-03T19:04:03.790+0000 7f0c622a5700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:32.788316+0000 front 2024-05-03T19:03:32.788339+0000 (oldest deadline 2024-05-03T19:03:58.088537+0000) 2024-05-03T19:04:04.052 INFO:journalctl@ceph.osd.5.smithi113.stdout:May 03 19:04:03 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-5[45660]: 2024-05-03T19:04:03.790+0000 7f0c622a5700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:38.589504+0000 front 2024-05-03T19:03:38.589555+0000 (oldest deadline 2024-05-03T19:04:02.089599+0000) 2024-05-03T19:04:04.052 INFO:journalctl@ceph.osd.8.smithi113.stdout:May 03 19:04:03 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-8[52667]: 2024-05-03T19:04:03.971+0000 7fb4a8e32700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:21.503361+0000 front 2024-05-03T19:03:21.503321+0000 (oldest deadline 2024-05-03T19:03:49.704026+0000) 2024-05-03T19:04:04.052 INFO:journalctl@ceph.osd.8.smithi113.stdout:May 03 19:04:03 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-8[52667]: 2024-05-03T19:04:03.971+0000 7fb4a8e32700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:30.805173+0000 front 2024-05-03T19:03:30.805157+0000 (oldest deadline 2024-05-03T19:03:53.605897+0000) 2024-05-03T19:04:04.052 INFO:journalctl@ceph.osd.8.smithi113.stdout:May 03 19:04:03 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-8[52667]: 2024-05-03T19:04:03.971+0000 7fb4a8e32700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:37.106650+0000 front 2024-05-03T19:03:37.106721+0000 (oldest deadline 2024-05-03T19:04:02.406804+0000) 2024-05-03T19:04:04.248 INFO:journalctl@ceph.osd.7.smithi045.stdout:May 03 19:04:03 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-7[51710]: 2024-05-03T19:04:03.778+0000 7fe2c0194700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:32.077137+0000 front 2024-05-03T19:03:32.077303+0000 (oldest deadline 2024-05-03T19:03:53.777433+0000) 2024-05-03T19:04:04.318 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:03 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:04:03.966+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:23.927793+0000 front 2024-05-03T19:03:23.927723+0000 (oldest deadline 2024-05-03T19:03:46.227921+0000) 2024-05-03T19:04:04.319 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:03 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:04:03.966+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:29.729118+0000 front 2024-05-03T19:03:29.729178+0000 (oldest deadline 2024-05-03T19:03:54.429041+0000) 2024-05-03T19:04:04.319 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:03 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:04:03.966+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:40.231110+0000 front 2024-05-03T19:03:40.230916+0000 (oldest deadline 2024-05-03T19:04:03.130962+0000) 2024-05-03T19:04:04.551 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:04 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:04.204+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:25.060144+0000 front 2024-05-03T19:03:25.060125+0000 (oldest deadline 2024-05-03T19:03:46.760321+0000) 2024-05-03T19:04:04.551 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:04 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:04.204+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:31.362284+0000 front 2024-05-03T19:03:31.362156+0000 (oldest deadline 2024-05-03T19:03:56.662340+0000) 2024-05-03T19:04:04.552 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:04 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:04.204+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:40.163402+0000 front 2024-05-03T19:03:40.163384+0000 (oldest deadline 2024-05-03T19:04:03.663534+0000) 2024-05-03T19:04:04.748 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:04 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:04.477+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:25.050521+0000 front 2024-05-03T19:03:25.050536+0000 (oldest deadline 2024-05-03T19:03:49.150320+0000) 2024-05-03T19:04:04.748 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:04 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:04.477+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:29.150598+0000 front 2024-05-03T19:03:29.150627+0000 (oldest deadline 2024-05-03T19:03:55.050741+0000) 2024-05-03T19:04:04.749 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:04 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:04.477+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:37.951720+0000 front 2024-05-03T19:03:37.951864+0000 (oldest deadline 2024-05-03T19:04:02.651644+0000) 2024-05-03T19:04:04.818 INFO:journalctl@ceph.osd.6.smithi032.stdout:May 03 19:04:04 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-6[55806]: 2024-05-03T19:04:04.347+0000 7f7d9b701700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:21.662275+0000 front 2024-05-03T19:03:21.662309+0000 (oldest deadline 2024-05-03T19:03:46.962364+0000) 2024-05-03T19:04:04.818 INFO:journalctl@ceph.osd.6.smithi032.stdout:May 03 19:04:04 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-6[55806]: 2024-05-03T19:04:04.347+0000 7f7d9b701700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:32.863448+0000 front 2024-05-03T19:03:32.863665+0000 (oldest deadline 2024-05-03T19:03:57.563487+0000) 2024-05-03T19:04:05.051 INFO:journalctl@ceph.osd.5.smithi113.stdout:May 03 19:04:04 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-5[45660]: 2024-05-03T19:04:04.756+0000 7f0c622a5700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:24.587577+0000 front 2024-05-03T19:03:24.587593+0000 (oldest deadline 2024-05-03T19:03:50.487474+0000) 2024-05-03T19:04:05.052 INFO:journalctl@ceph.osd.5.smithi113.stdout:May 03 19:04:04 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-5[45660]: 2024-05-03T19:04:04.756+0000 7f0c622a5700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:32.788316+0000 front 2024-05-03T19:03:32.788339+0000 (oldest deadline 2024-05-03T19:03:58.088537+0000) 2024-05-03T19:04:05.052 INFO:journalctl@ceph.osd.5.smithi113.stdout:May 03 19:04:04 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-5[45660]: 2024-05-03T19:04:04.756+0000 7f0c622a5700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:38.589504+0000 front 2024-05-03T19:03:38.589555+0000 (oldest deadline 2024-05-03T19:04:02.089599+0000) 2024-05-03T19:04:05.052 INFO:journalctl@ceph.osd.8.smithi113.stdout:May 03 19:04:04 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-8[52667]: 2024-05-03T19:04:04.956+0000 7fb4a8e32700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:21.503361+0000 front 2024-05-03T19:03:21.503321+0000 (oldest deadline 2024-05-03T19:03:49.704026+0000) 2024-05-03T19:04:05.052 INFO:journalctl@ceph.osd.8.smithi113.stdout:May 03 19:04:04 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-8[52667]: 2024-05-03T19:04:04.956+0000 7fb4a8e32700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:30.805173+0000 front 2024-05-03T19:03:30.805157+0000 (oldest deadline 2024-05-03T19:03:53.605897+0000) 2024-05-03T19:04:05.052 INFO:journalctl@ceph.osd.8.smithi113.stdout:May 03 19:04:04 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-8[52667]: 2024-05-03T19:04:04.956+0000 7fb4a8e32700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:37.106650+0000 front 2024-05-03T19:03:37.106721+0000 (oldest deadline 2024-05-03T19:04:02.406804+0000) 2024-05-03T19:04:05.248 INFO:journalctl@ceph.osd.7.smithi045.stdout:May 03 19:04:04 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-7[51710]: 2024-05-03T19:04:04.787+0000 7fe2c0194700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:32.077137+0000 front 2024-05-03T19:03:32.077303+0000 (oldest deadline 2024-05-03T19:03:53.777433+0000) 2024-05-03T19:04:05.322 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:04 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:04:04.997+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:23.927793+0000 front 2024-05-03T19:03:23.927723+0000 (oldest deadline 2024-05-03T19:03:46.227921+0000) 2024-05-03T19:04:05.322 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:04 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:04:04.997+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:29.729118+0000 front 2024-05-03T19:03:29.729178+0000 (oldest deadline 2024-05-03T19:03:54.429041+0000) 2024-05-03T19:04:05.322 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:04 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:04:04.997+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:40.231110+0000 front 2024-05-03T19:03:40.230916+0000 (oldest deadline 2024-05-03T19:04:03.130962+0000) 2024-05-03T19:04:05.551 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:05 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:05.179+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:25.060144+0000 front 2024-05-03T19:03:25.060125+0000 (oldest deadline 2024-05-03T19:03:46.760321+0000) 2024-05-03T19:04:05.552 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:05 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:05.179+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:31.362284+0000 front 2024-05-03T19:03:31.362156+0000 (oldest deadline 2024-05-03T19:03:56.662340+0000) 2024-05-03T19:04:05.552 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:05 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:05.179+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:40.163402+0000 front 2024-05-03T19:03:40.163384+0000 (oldest deadline 2024-05-03T19:04:03.663534+0000) 2024-05-03T19:04:05.748 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:05 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:05.489+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:25.050521+0000 front 2024-05-03T19:03:25.050536+0000 (oldest deadline 2024-05-03T19:03:49.150320+0000) 2024-05-03T19:04:05.748 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:05 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:05.489+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:29.150598+0000 front 2024-05-03T19:03:29.150627+0000 (oldest deadline 2024-05-03T19:03:55.050741+0000) 2024-05-03T19:04:05.749 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:05 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:05.489+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:37.951720+0000 front 2024-05-03T19:03:37.951864+0000 (oldest deadline 2024-05-03T19:04:02.651644+0000) 2024-05-03T19:04:05.819 INFO:journalctl@ceph.osd.6.smithi032.stdout:May 03 19:04:05 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-6[55806]: 2024-05-03T19:04:05.362+0000 7f7d9b701700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:21.662275+0000 front 2024-05-03T19:03:21.662309+0000 (oldest deadline 2024-05-03T19:03:46.962364+0000) 2024-05-03T19:04:05.819 INFO:journalctl@ceph.osd.6.smithi032.stdout:May 03 19:04:05 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-6[55806]: 2024-05-03T19:04:05.362+0000 7f7d9b701700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:32.863448+0000 front 2024-05-03T19:03:32.863665+0000 (oldest deadline 2024-05-03T19:03:57.563487+0000) 2024-05-03T19:04:05.819 INFO:journalctl@ceph.osd.6.smithi032.stdout:May 03 19:04:05 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-6[55806]: 2024-05-03T19:04:05.362+0000 7f7d9b701700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:40.464732+0000 front 2024-05-03T19:03:40.464684+0000 (oldest deadline 2024-05-03T19:04:05.164736+0000) 2024-05-03T19:04:06.248 INFO:journalctl@ceph.osd.7.smithi045.stdout:May 03 19:04:05 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-7[51710]: 2024-05-03T19:04:05.781+0000 7fe2c0194700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:32.077137+0000 front 2024-05-03T19:03:32.077303+0000 (oldest deadline 2024-05-03T19:03:53.777433+0000) 2024-05-03T19:04:06.248 INFO:journalctl@ceph.osd.7.smithi045.stdout:May 03 19:04:05 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-7[51710]: 2024-05-03T19:04:05.781+0000 7fe2c0194700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:39.579099+0000 front 2024-05-03T19:03:39.579074+0000 (oldest deadline 2024-05-03T19:04:05.479057+0000) 2024-05-03T19:04:06.273 INFO:journalctl@ceph.osd.5.smithi113.stdout:May 03 19:04:05 smithi113 podman[70289]: 2024-05-03 19:04:05.99393647 +0000 UTC m=+5.605018029 container died ac8bbef59b6ed0726f66b6b5ef5a4fc063ef04afa2de48fd95768605578e5b55 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.build-date=20240430, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-05-03T19:04:06.274 INFO:journalctl@ceph.osd.8.smithi113.stdout:May 03 19:04:05 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-8[52667]: 2024-05-03T19:04:05.918+0000 7fb4a8e32700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:21.503361+0000 front 2024-05-03T19:03:21.503321+0000 (oldest deadline 2024-05-03T19:03:49.704026+0000) 2024-05-03T19:04:06.274 INFO:journalctl@ceph.osd.8.smithi113.stdout:May 03 19:04:05 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-8[52667]: 2024-05-03T19:04:05.918+0000 7fb4a8e32700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:30.805173+0000 front 2024-05-03T19:03:30.805157+0000 (oldest deadline 2024-05-03T19:03:53.605897+0000) 2024-05-03T19:04:06.274 INFO:journalctl@ceph.osd.8.smithi113.stdout:May 03 19:04:05 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-8[52667]: 2024-05-03T19:04:05.918+0000 7fb4a8e32700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:37.106650+0000 front 2024-05-03T19:03:37.106721+0000 (oldest deadline 2024-05-03T19:04:02.406804+0000) 2024-05-03T19:04:06.274 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:06 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:06.151+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:25.060144+0000 front 2024-05-03T19:03:25.060125+0000 (oldest deadline 2024-05-03T19:03:46.760321+0000) 2024-05-03T19:04:06.274 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:06 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:06.151+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:31.362284+0000 front 2024-05-03T19:03:31.362156+0000 (oldest deadline 2024-05-03T19:03:56.662340+0000) 2024-05-03T19:04:06.274 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:06 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:06.151+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:40.163402+0000 front 2024-05-03T19:03:40.163384+0000 (oldest deadline 2024-05-03T19:04:03.663534+0000) 2024-05-03T19:04:06.319 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:06 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:04:06.018+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:23.927793+0000 front 2024-05-03T19:03:23.927723+0000 (oldest deadline 2024-05-03T19:03:46.227921+0000) 2024-05-03T19:04:06.319 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:06 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:04:06.018+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:29.729118+0000 front 2024-05-03T19:03:29.729178+0000 (oldest deadline 2024-05-03T19:03:54.429041+0000) 2024-05-03T19:04:06.319 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:06 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:04:06.018+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:40.231110+0000 front 2024-05-03T19:03:40.230916+0000 (oldest deadline 2024-05-03T19:04:03.130962+0000) 2024-05-03T19:04:06.551 INFO:journalctl@ceph.osd.5.smithi113.stdout:May 03 19:04:06 smithi113 podman[70289]: 2024-05-03 19:04:06.371932584 +0000 UTC m=+5.983014144 container cleanup ac8bbef59b6ed0726f66b6b5ef5a4fc063ef04afa2de48fd95768605578e5b55 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-5, org.label-schema.build-date=20240430, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-05-03T19:04:06.551 INFO:journalctl@ceph.osd.5.smithi113.stdout:May 03 19:04:06 smithi113 bash[70289]: ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-5 2024-05-03T19:04:06.782 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:06 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:06.497+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:25.050521+0000 front 2024-05-03T19:03:25.050536+0000 (oldest deadline 2024-05-03T19:03:49.150320+0000) 2024-05-03T19:04:06.782 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:06 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:06.497+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:29.150598+0000 front 2024-05-03T19:03:29.150627+0000 (oldest deadline 2024-05-03T19:03:55.050741+0000) 2024-05-03T19:04:06.782 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:06 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:06.497+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:37.951720+0000 front 2024-05-03T19:03:37.951864+0000 (oldest deadline 2024-05-03T19:04:02.651644+0000) 2024-05-03T19:04:06.818 INFO:journalctl@ceph.osd.6.smithi032.stdout:May 03 19:04:06 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-6[55806]: 2024-05-03T19:04:06.353+0000 7f7d9b701700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:21.662275+0000 front 2024-05-03T19:03:21.662309+0000 (oldest deadline 2024-05-03T19:03:46.962364+0000) 2024-05-03T19:04:06.819 INFO:journalctl@ceph.osd.6.smithi032.stdout:May 03 19:04:06 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-6[55806]: 2024-05-03T19:04:06.353+0000 7f7d9b701700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:32.863448+0000 front 2024-05-03T19:03:32.863665+0000 (oldest deadline 2024-05-03T19:03:57.563487+0000) 2024-05-03T19:04:06.819 INFO:journalctl@ceph.osd.6.smithi032.stdout:May 03 19:04:06 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-6[55806]: 2024-05-03T19:04:06.353+0000 7f7d9b701700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:40.464732+0000 front 2024-05-03T19:03:40.464684+0000 (oldest deadline 2024-05-03T19:04:05.164736+0000) 2024-05-03T19:04:06.932 INFO:journalctl@ceph.osd.5.smithi113.stdout:May 03 19:04:06 smithi113 podman[70304]: 2024-05-03 19:04:06.640028839 +0000 UTC m=+0.643943492 container remove ac8bbef59b6ed0726f66b6b5ef5a4fc063ef04afa2de48fd95768605578e5b55 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240430, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe) 2024-05-03T19:04:07.183 INFO:journalctl@ceph.osd.8.smithi113.stdout:May 03 19:04:06 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-8[52667]: 2024-05-03T19:04:06.931+0000 7fb4a8e32700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:21.503361+0000 front 2024-05-03T19:03:21.503321+0000 (oldest deadline 2024-05-03T19:03:49.704026+0000) 2024-05-03T19:04:07.183 INFO:journalctl@ceph.osd.8.smithi113.stdout:May 03 19:04:06 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-8[52667]: 2024-05-03T19:04:06.931+0000 7fb4a8e32700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:30.805173+0000 front 2024-05-03T19:03:30.805157+0000 (oldest deadline 2024-05-03T19:03:53.605897+0000) 2024-05-03T19:04:07.183 INFO:journalctl@ceph.osd.8.smithi113.stdout:May 03 19:04:06 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-8[52667]: 2024-05-03T19:04:06.931+0000 7fb4a8e32700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:37.106650+0000 front 2024-05-03T19:03:37.106721+0000 (oldest deadline 2024-05-03T19:04:02.406804+0000) 2024-05-03T19:04:07.184 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:07 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:07.181+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:25.060144+0000 front 2024-05-03T19:03:25.060125+0000 (oldest deadline 2024-05-03T19:03:46.760321+0000) 2024-05-03T19:04:07.184 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:07 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:07.181+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:31.362284+0000 front 2024-05-03T19:03:31.362156+0000 (oldest deadline 2024-05-03T19:03:56.662340+0000) 2024-05-03T19:04:07.184 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:07 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:07.181+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:40.163402+0000 front 2024-05-03T19:03:40.163384+0000 (oldest deadline 2024-05-03T19:04:03.663534+0000) 2024-05-03T19:04:07.184 INFO:journalctl@ceph.osd.5.smithi113.stdout:May 03 19:04:06 smithi113 podman[70398]: 2024-05-03 19:04:06.932327427 +0000 UTC m=+0.111936903 container create dba05408c6924af211f2393ca5917c3b05f8be2e23d1b79863d6888eff8c7d03 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-5-deactivate, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240430, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-05-03T19:04:07.184 INFO:journalctl@ceph.osd.5.smithi113.stdout:May 03 19:04:06 smithi113 podman[70398]: 2024-05-03 19:04:06.838438397 +0000 UTC m=+0.018047896 image pull ee4d846202496ba6096593e8e8becc0700a5eb7ebd95536031187f5f337d4012 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd 2024-05-03T19:04:07.185 INFO:journalctl@ceph.osd.5.smithi113.stdout:May 03 19:04:07 smithi113 podman[70398]: 2024-05-03 19:04:07.159926517 +0000 UTC m=+0.339535993 container init dba05408c6924af211f2393ca5917c3b05f8be2e23d1b79863d6888eff8c7d03 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-5-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240430, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.license=GPLv2) 2024-05-03T19:04:07.185 INFO:journalctl@ceph.osd.5.smithi113.stdout:May 03 19:04:07 smithi113 podman[70398]: 2024-05-03 19:04:07.16300444 +0000 UTC m=+0.342613916 container start dba05408c6924af211f2393ca5917c3b05f8be2e23d1b79863d6888eff8c7d03 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-5-deactivate, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240430, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-03T19:04:07.248 INFO:journalctl@ceph.osd.7.smithi045.stdout:May 03 19:04:06 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-7[51710]: 2024-05-03T19:04:06.780+0000 7fe2c0194700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:32.077137+0000 front 2024-05-03T19:03:32.077303+0000 (oldest deadline 2024-05-03T19:03:53.777433+0000) 2024-05-03T19:04:07.248 INFO:journalctl@ceph.osd.7.smithi045.stdout:May 03 19:04:06 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-7[51710]: 2024-05-03T19:04:06.780+0000 7fe2c0194700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:39.579099+0000 front 2024-05-03T19:03:39.579074+0000 (oldest deadline 2024-05-03T19:04:05.479057+0000) 2024-05-03T19:04:07.318 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:06 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:04:06.996+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:23.927793+0000 front 2024-05-03T19:03:23.927723+0000 (oldest deadline 2024-05-03T19:03:46.227921+0000) 2024-05-03T19:04:07.319 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:06 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:04:06.996+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:29.729118+0000 front 2024-05-03T19:03:29.729178+0000 (oldest deadline 2024-05-03T19:03:54.429041+0000) 2024-05-03T19:04:07.319 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:06 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:04:06.996+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:40.231110+0000 front 2024-05-03T19:03:40.230916+0000 (oldest deadline 2024-05-03T19:04:03.130962+0000) 2024-05-03T19:04:07.450 INFO:journalctl@ceph.osd.5.smithi113.stdout:May 03 19:04:07 smithi113 podman[70398]: 2024-05-03 19:04:07.260658765 +0000 UTC m=+0.440268243 container attach dba05408c6924af211f2393ca5917c3b05f8be2e23d1b79863d6888eff8c7d03 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-5-deactivate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240430, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True) 2024-05-03T19:04:07.617 INFO:journalctl@ceph.osd.6.smithi032.stdout:May 03 19:04:07 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-6[55806]: 2024-05-03T19:04:07.364+0000 7f7d9b701700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:21.662275+0000 front 2024-05-03T19:03:21.662309+0000 (oldest deadline 2024-05-03T19:03:46.962364+0000) 2024-05-03T19:04:07.617 INFO:journalctl@ceph.osd.6.smithi032.stdout:May 03 19:04:07 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-6[55806]: 2024-05-03T19:04:07.364+0000 7f7d9b701700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:32.863448+0000 front 2024-05-03T19:03:32.863665+0000 (oldest deadline 2024-05-03T19:03:57.563487+0000) 2024-05-03T19:04:07.617 INFO:journalctl@ceph.osd.6.smithi032.stdout:May 03 19:04:07 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-6[55806]: 2024-05-03T19:04:07.364+0000 7f7d9b701700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:40.464732+0000 front 2024-05-03T19:03:40.464684+0000 (oldest deadline 2024-05-03T19:04:05.164736+0000) 2024-05-03T19:04:07.730 INFO:journalctl@ceph.osd.5.smithi113.stdout:May 03 19:04:07 smithi113 podman[70398]: 2024-05-03 19:04:07.450218182 +0000 UTC m=+0.629827670 container died dba05408c6924af211f2393ca5917c3b05f8be2e23d1b79863d6888eff8c7d03 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240430, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-05-03T19:04:07.808 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:07 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:07.534+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:25.050521+0000 front 2024-05-03T19:03:25.050536+0000 (oldest deadline 2024-05-03T19:03:49.150320+0000) 2024-05-03T19:04:07.808 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:07 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:07.534+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:29.150598+0000 front 2024-05-03T19:03:29.150627+0000 (oldest deadline 2024-05-03T19:03:55.050741+0000) 2024-05-03T19:04:07.808 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:07 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:07.534+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:37.951720+0000 front 2024-05-03T19:03:37.951864+0000 (oldest deadline 2024-05-03T19:04:02.651644+0000) 2024-05-03T19:04:07.970 DEBUG:teuthology.orchestra.run.smithi113:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f500455e-097b-11ef-bc97-c7b262605968@osd.5.service' 2024-05-03T19:04:08.035 INFO:journalctl@ceph.osd.8.smithi113.stdout:May 03 19:04:07 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-8[52667]: 2024-05-03T19:04:07.966+0000 7fb4a8e32700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:21.503361+0000 front 2024-05-03T19:03:21.503321+0000 (oldest deadline 2024-05-03T19:03:49.704026+0000) 2024-05-03T19:04:08.035 INFO:journalctl@ceph.osd.8.smithi113.stdout:May 03 19:04:07 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-8[52667]: 2024-05-03T19:04:07.966+0000 7fb4a8e32700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:30.805173+0000 front 2024-05-03T19:03:30.805157+0000 (oldest deadline 2024-05-03T19:03:53.605897+0000) 2024-05-03T19:04:08.035 INFO:journalctl@ceph.osd.8.smithi113.stdout:May 03 19:04:07 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-8[52667]: 2024-05-03T19:04:07.966+0000 7fb4a8e32700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:37.106650+0000 front 2024-05-03T19:03:37.106721+0000 (oldest deadline 2024-05-03T19:04:02.406804+0000) 2024-05-03T19:04:08.035 INFO:journalctl@ceph.osd.5.smithi113.stdout:May 03 19:04:07 smithi113 podman[70423]: 2024-05-03 19:04:07.947472685 +0000 UTC m=+0.492378596 container remove dba05408c6924af211f2393ca5917c3b05f8be2e23d1b79863d6888eff8c7d03 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-5-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240430, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-05-03T19:04:08.035 INFO:journalctl@ceph.osd.5.smithi113.stdout:May 03 19:04:07 smithi113 systemd[1]: ceph-f500455e-097b-11ef-bc97-c7b262605968@osd.5.service: Deactivated successfully. 2024-05-03T19:04:08.036 INFO:journalctl@ceph.osd.5.smithi113.stdout:May 03 19:04:07 smithi113 systemd[1]: Stopped Ceph osd.5 for f500455e-097b-11ef-bc97-c7b262605968. 2024-05-03T19:04:08.036 INFO:journalctl@ceph.osd.5.smithi113.stdout:May 03 19:04:07 smithi113 systemd[1]: ceph-f500455e-097b-11ef-bc97-c7b262605968@osd.5.service: Consumed 11.962s CPU time. 2024-05-03T19:04:08.051 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-03T19:04:08.051 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-05-03T19:04:08.051 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-05-03T19:04:08.051 DEBUG:teuthology.orchestra.run.smithi032:> sudo systemctl stop ceph-f500455e-097b-11ef-bc97-c7b262605968@osd.6 2024-05-03T19:04:08.248 INFO:journalctl@ceph.osd.7.smithi045.stdout:May 03 19:04:07 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-7[51710]: 2024-05-03T19:04:07.806+0000 7fe2c0194700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:32.077137+0000 front 2024-05-03T19:03:32.077303+0000 (oldest deadline 2024-05-03T19:03:53.777433+0000) 2024-05-03T19:04:08.274 INFO:journalctl@ceph.osd.7.smithi045.stdout:May 03 19:04:07 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-7[51710]: 2024-05-03T19:04:07.806+0000 7fe2c0194700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:39.579099+0000 front 2024-05-03T19:03:39.579074+0000 (oldest deadline 2024-05-03T19:04:05.479057+0000) 2024-05-03T19:04:08.301 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:08 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:08.202+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:25.060144+0000 front 2024-05-03T19:03:25.060125+0000 (oldest deadline 2024-05-03T19:03:46.760321+0000) 2024-05-03T19:04:08.301 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:08 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:08.202+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:31.362284+0000 front 2024-05-03T19:03:31.362156+0000 (oldest deadline 2024-05-03T19:03:56.662340+0000) 2024-05-03T19:04:08.301 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:08 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:08.202+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:40.163402+0000 front 2024-05-03T19:03:40.163384+0000 (oldest deadline 2024-05-03T19:04:03.663534+0000) 2024-05-03T19:04:08.318 INFO:journalctl@ceph.osd.6.smithi032.stdout:May 03 19:04:08 smithi032 systemd[1]: Stopping Ceph osd.6 for f500455e-097b-11ef-bc97-c7b262605968... 2024-05-03T19:04:08.319 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:08 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:04:08.032+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:23.927793+0000 front 2024-05-03T19:03:23.927723+0000 (oldest deadline 2024-05-03T19:03:46.227921+0000) 2024-05-03T19:04:08.320 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:08 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:04:08.032+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:29.729118+0000 front 2024-05-03T19:03:29.729178+0000 (oldest deadline 2024-05-03T19:03:54.429041+0000) 2024-05-03T19:04:08.320 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:08 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:04:08.032+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:40.231110+0000 front 2024-05-03T19:03:40.230916+0000 (oldest deadline 2024-05-03T19:04:03.130962+0000) 2024-05-03T19:04:08.819 INFO:journalctl@ceph.osd.6.smithi032.stdout:May 03 19:04:08 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-6[55806]: 2024-05-03T19:04:08.373+0000 7f7d9f10d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-03T19:04:08.819 INFO:journalctl@ceph.osd.6.smithi032.stdout:May 03 19:04:08 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-6[55806]: 2024-05-03T19:04:08.373+0000 7f7d9f10d700 -1 osd.6 85 *** Got signal Terminated *** 2024-05-03T19:04:08.819 INFO:journalctl@ceph.osd.6.smithi032.stdout:May 03 19:04:08 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-6[55806]: 2024-05-03T19:04:08.373+0000 7f7d9f10d700 -1 osd.6 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-03T19:04:08.819 INFO:journalctl@ceph.osd.6.smithi032.stdout:May 03 19:04:08 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-6[55806]: 2024-05-03T19:04:08.411+0000 7f7d9b701700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:21.662275+0000 front 2024-05-03T19:03:21.662309+0000 (oldest deadline 2024-05-03T19:03:46.962364+0000) 2024-05-03T19:04:08.819 INFO:journalctl@ceph.osd.6.smithi032.stdout:May 03 19:04:08 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-6[55806]: 2024-05-03T19:04:08.411+0000 7f7d9b701700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:32.863448+0000 front 2024-05-03T19:03:32.863665+0000 (oldest deadline 2024-05-03T19:03:57.563487+0000) 2024-05-03T19:04:08.819 INFO:journalctl@ceph.osd.6.smithi032.stdout:May 03 19:04:08 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-6[55806]: 2024-05-03T19:04:08.411+0000 7f7d9b701700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:40.464732+0000 front 2024-05-03T19:03:40.464684+0000 (oldest deadline 2024-05-03T19:04:05.164736+0000) 2024-05-03T19:04:08.998 INFO:journalctl@ceph.osd.7.smithi045.stdout:May 03 19:04:08 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-7[51710]: 2024-05-03T19:04:08.822+0000 7fe2c0194700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:32.077137+0000 front 2024-05-03T19:03:32.077303+0000 (oldest deadline 2024-05-03T19:03:53.777433+0000) 2024-05-03T19:04:08.998 INFO:journalctl@ceph.osd.7.smithi045.stdout:May 03 19:04:08 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-7[51710]: 2024-05-03T19:04:08.822+0000 7fe2c0194700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:39.579099+0000 front 2024-05-03T19:03:39.579074+0000 (oldest deadline 2024-05-03T19:04:05.479057+0000) 2024-05-03T19:04:08.999 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:08 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:08.575+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:25.050521+0000 front 2024-05-03T19:03:25.050536+0000 (oldest deadline 2024-05-03T19:03:49.150320+0000) 2024-05-03T19:04:08.999 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:08 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:08.575+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:29.150598+0000 front 2024-05-03T19:03:29.150627+0000 (oldest deadline 2024-05-03T19:03:55.050741+0000) 2024-05-03T19:04:08.999 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:08 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:08.575+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:37.951720+0000 front 2024-05-03T19:03:37.951864+0000 (oldest deadline 2024-05-03T19:04:02.651644+0000) 2024-05-03T19:04:09.302 INFO:journalctl@ceph.osd.8.smithi113.stdout:May 03 19:04:09 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-8[52667]: 2024-05-03T19:04:09.002+0000 7fb4a8e32700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:21.503361+0000 front 2024-05-03T19:03:21.503321+0000 (oldest deadline 2024-05-03T19:03:49.704026+0000) 2024-05-03T19:04:09.302 INFO:journalctl@ceph.osd.8.smithi113.stdout:May 03 19:04:09 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-8[52667]: 2024-05-03T19:04:09.003+0000 7fb4a8e32700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:30.805173+0000 front 2024-05-03T19:03:30.805157+0000 (oldest deadline 2024-05-03T19:03:53.605897+0000) 2024-05-03T19:04:09.303 INFO:journalctl@ceph.osd.8.smithi113.stdout:May 03 19:04:09 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-8[52667]: 2024-05-03T19:04:09.003+0000 7fb4a8e32700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:37.106650+0000 front 2024-05-03T19:03:37.106721+0000 (oldest deadline 2024-05-03T19:04:02.406804+0000) 2024-05-03T19:04:09.303 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:09 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:09.199+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:25.060144+0000 front 2024-05-03T19:03:25.060125+0000 (oldest deadline 2024-05-03T19:03:46.760321+0000) 2024-05-03T19:04:09.303 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:09 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:09.199+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:31.362284+0000 front 2024-05-03T19:03:31.362156+0000 (oldest deadline 2024-05-03T19:03:56.662340+0000) 2024-05-03T19:04:09.304 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:09 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:09.199+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:40.163402+0000 front 2024-05-03T19:03:40.163384+0000 (oldest deadline 2024-05-03T19:04:03.663534+0000) 2024-05-03T19:04:09.318 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:08 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:04:08.998+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:23.927793+0000 front 2024-05-03T19:03:23.927723+0000 (oldest deadline 2024-05-03T19:03:46.227921+0000) 2024-05-03T19:04:09.319 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:08 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:04:08.998+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:29.729118+0000 front 2024-05-03T19:03:29.729178+0000 (oldest deadline 2024-05-03T19:03:54.429041+0000) 2024-05-03T19:04:09.319 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:08 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:04:08.998+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:40.231110+0000 front 2024-05-03T19:03:40.230916+0000 (oldest deadline 2024-05-03T19:04:03.130962+0000) 2024-05-03T19:04:09.819 INFO:journalctl@ceph.osd.6.smithi032.stdout:May 03 19:04:09 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-6[55806]: 2024-05-03T19:04:09.411+0000 7f7d9b701700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:21.662275+0000 front 2024-05-03T19:03:21.662309+0000 (oldest deadline 2024-05-03T19:03:46.962364+0000) 2024-05-03T19:04:09.819 INFO:journalctl@ceph.osd.6.smithi032.stdout:May 03 19:04:09 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-6[55806]: 2024-05-03T19:04:09.411+0000 7f7d9b701700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:32.863448+0000 front 2024-05-03T19:03:32.863665+0000 (oldest deadline 2024-05-03T19:03:57.563487+0000) 2024-05-03T19:04:09.819 INFO:journalctl@ceph.osd.6.smithi032.stdout:May 03 19:04:09 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-6[55806]: 2024-05-03T19:04:09.411+0000 7f7d9b701700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:40.464732+0000 front 2024-05-03T19:03:40.464684+0000 (oldest deadline 2024-05-03T19:04:05.164736+0000) 2024-05-03T19:04:09.998 INFO:journalctl@ceph.osd.7.smithi045.stdout:May 03 19:04:09 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-7[51710]: 2024-05-03T19:04:09.828+0000 7fe2c0194700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:32.077137+0000 front 2024-05-03T19:03:32.077303+0000 (oldest deadline 2024-05-03T19:03:53.777433+0000) 2024-05-03T19:04:09.998 INFO:journalctl@ceph.osd.7.smithi045.stdout:May 03 19:04:09 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-7[51710]: 2024-05-03T19:04:09.828+0000 7fe2c0194700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:39.579099+0000 front 2024-05-03T19:03:39.579074+0000 (oldest deadline 2024-05-03T19:04:05.479057+0000) 2024-05-03T19:04:09.998 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:09 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:09.623+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:25.050521+0000 front 2024-05-03T19:03:25.050536+0000 (oldest deadline 2024-05-03T19:03:49.150320+0000) 2024-05-03T19:04:09.999 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:09 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:09.623+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:29.150598+0000 front 2024-05-03T19:03:29.150627+0000 (oldest deadline 2024-05-03T19:03:55.050741+0000) 2024-05-03T19:04:09.999 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:09 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:09.623+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:37.951720+0000 front 2024-05-03T19:03:37.951864+0000 (oldest deadline 2024-05-03T19:04:02.651644+0000) 2024-05-03T19:04:10.301 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:10 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:10.212+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:25.060144+0000 front 2024-05-03T19:03:25.060125+0000 (oldest deadline 2024-05-03T19:03:46.760321+0000) 2024-05-03T19:04:10.302 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:10 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:10.212+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:31.362284+0000 front 2024-05-03T19:03:31.362156+0000 (oldest deadline 2024-05-03T19:03:56.662340+0000) 2024-05-03T19:04:10.302 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:10 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:10.212+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:40.163402+0000 front 2024-05-03T19:03:40.163384+0000 (oldest deadline 2024-05-03T19:04:03.663534+0000) 2024-05-03T19:04:10.302 INFO:journalctl@ceph.osd.8.smithi113.stdout:May 03 19:04:09 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-8[52667]: 2024-05-03T19:04:09.983+0000 7fb4a8e32700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:21.503361+0000 front 2024-05-03T19:03:21.503321+0000 (oldest deadline 2024-05-03T19:03:49.704026+0000) 2024-05-03T19:04:10.302 INFO:journalctl@ceph.osd.8.smithi113.stdout:May 03 19:04:09 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-8[52667]: 2024-05-03T19:04:09.983+0000 7fb4a8e32700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:30.805173+0000 front 2024-05-03T19:03:30.805157+0000 (oldest deadline 2024-05-03T19:03:53.605897+0000) 2024-05-03T19:04:10.302 INFO:journalctl@ceph.osd.8.smithi113.stdout:May 03 19:04:09 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-8[52667]: 2024-05-03T19:04:09.983+0000 7fb4a8e32700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:37.106650+0000 front 2024-05-03T19:03:37.106721+0000 (oldest deadline 2024-05-03T19:04:02.406804+0000) 2024-05-03T19:04:10.318 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:09 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:04:09.962+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:23.927793+0000 front 2024-05-03T19:03:23.927723+0000 (oldest deadline 2024-05-03T19:03:46.227921+0000) 2024-05-03T19:04:10.318 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:09 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:04:09.962+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:29.729118+0000 front 2024-05-03T19:03:29.729178+0000 (oldest deadline 2024-05-03T19:03:54.429041+0000) 2024-05-03T19:04:10.318 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:09 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:04:09.962+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:40.231110+0000 front 2024-05-03T19:03:40.230916+0000 (oldest deadline 2024-05-03T19:04:03.130962+0000) 2024-05-03T19:04:10.818 INFO:journalctl@ceph.osd.6.smithi032.stdout:May 03 19:04:10 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-6[55806]: 2024-05-03T19:04:10.428+0000 7f7d9b701700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:21.662275+0000 front 2024-05-03T19:03:21.662309+0000 (oldest deadline 2024-05-03T19:03:46.962364+0000) 2024-05-03T19:04:10.819 INFO:journalctl@ceph.osd.6.smithi032.stdout:May 03 19:04:10 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-6[55806]: 2024-05-03T19:04:10.428+0000 7f7d9b701700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:32.863448+0000 front 2024-05-03T19:03:32.863665+0000 (oldest deadline 2024-05-03T19:03:57.563487+0000) 2024-05-03T19:04:10.819 INFO:journalctl@ceph.osd.6.smithi032.stdout:May 03 19:04:10 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-6[55806]: 2024-05-03T19:04:10.428+0000 7f7d9b701700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:40.464732+0000 front 2024-05-03T19:03:40.464684+0000 (oldest deadline 2024-05-03T19:04:05.164736+0000) 2024-05-03T19:04:10.998 INFO:journalctl@ceph.osd.7.smithi045.stdout:May 03 19:04:10 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-7[51710]: 2024-05-03T19:04:10.831+0000 7fe2c0194700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:32.077137+0000 front 2024-05-03T19:03:32.077303+0000 (oldest deadline 2024-05-03T19:03:53.777433+0000) 2024-05-03T19:04:10.999 INFO:journalctl@ceph.osd.7.smithi045.stdout:May 03 19:04:10 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-7[51710]: 2024-05-03T19:04:10.831+0000 7fe2c0194700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:39.579099+0000 front 2024-05-03T19:03:39.579074+0000 (oldest deadline 2024-05-03T19:04:05.479057+0000) 2024-05-03T19:04:10.999 INFO:journalctl@ceph.osd.7.smithi045.stdout:May 03 19:04:10 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-7[51710]: 2024-05-03T19:04:10.831+0000 7fe2c0194700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-05-03T19:03:45.479498+0000 front 2024-05-03T19:03:45.479594+0000 (oldest deadline 2024-05-03T19:04:10.779647+0000) 2024-05-03T19:04:10.999 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:10 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:10.602+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:25.050521+0000 front 2024-05-03T19:03:25.050536+0000 (oldest deadline 2024-05-03T19:03:49.150320+0000) 2024-05-03T19:04:10.999 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:10 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:10.602+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:29.150598+0000 front 2024-05-03T19:03:29.150627+0000 (oldest deadline 2024-05-03T19:03:55.050741+0000) 2024-05-03T19:04:10.999 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:10 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:10.602+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:37.951720+0000 front 2024-05-03T19:03:37.951864+0000 (oldest deadline 2024-05-03T19:04:02.651644+0000) 2024-05-03T19:04:11.301 INFO:journalctl@ceph.osd.8.smithi113.stdout:May 03 19:04:10 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-8[52667]: 2024-05-03T19:04:10.972+0000 7fb4a8e32700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:21.503361+0000 front 2024-05-03T19:03:21.503321+0000 (oldest deadline 2024-05-03T19:03:49.704026+0000) 2024-05-03T19:04:11.302 INFO:journalctl@ceph.osd.8.smithi113.stdout:May 03 19:04:10 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-8[52667]: 2024-05-03T19:04:10.972+0000 7fb4a8e32700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:30.805173+0000 front 2024-05-03T19:03:30.805157+0000 (oldest deadline 2024-05-03T19:03:53.605897+0000) 2024-05-03T19:04:11.302 INFO:journalctl@ceph.osd.8.smithi113.stdout:May 03 19:04:10 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-8[52667]: 2024-05-03T19:04:10.972+0000 7fb4a8e32700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:37.106650+0000 front 2024-05-03T19:03:37.106721+0000 (oldest deadline 2024-05-03T19:04:02.406804+0000) 2024-05-03T19:04:11.302 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:11 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:11.200+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:25.060144+0000 front 2024-05-03T19:03:25.060125+0000 (oldest deadline 2024-05-03T19:03:46.760321+0000) 2024-05-03T19:04:11.302 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:11 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:11.200+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:31.362284+0000 front 2024-05-03T19:03:31.362156+0000 (oldest deadline 2024-05-03T19:03:56.662340+0000) 2024-05-03T19:04:11.303 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:11 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:11.200+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:40.163402+0000 front 2024-05-03T19:03:40.163384+0000 (oldest deadline 2024-05-03T19:04:03.663534+0000) 2024-05-03T19:04:11.318 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:11 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:04:11.000+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:23.927793+0000 front 2024-05-03T19:03:23.927723+0000 (oldest deadline 2024-05-03T19:03:46.227921+0000) 2024-05-03T19:04:11.318 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:11 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:04:11.000+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:29.729118+0000 front 2024-05-03T19:03:29.729178+0000 (oldest deadline 2024-05-03T19:03:54.429041+0000) 2024-05-03T19:04:11.319 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:11 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:04:11.000+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:40.231110+0000 front 2024-05-03T19:03:40.230916+0000 (oldest deadline 2024-05-03T19:04:03.130962+0000) 2024-05-03T19:04:11.818 INFO:journalctl@ceph.osd.6.smithi032.stdout:May 03 19:04:11 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-6[55806]: 2024-05-03T19:04:11.394+0000 7f7d9b701700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:21.662275+0000 front 2024-05-03T19:03:21.662309+0000 (oldest deadline 2024-05-03T19:03:46.962364+0000) 2024-05-03T19:04:11.818 INFO:journalctl@ceph.osd.6.smithi032.stdout:May 03 19:04:11 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-6[55806]: 2024-05-03T19:04:11.394+0000 7f7d9b701700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:32.863448+0000 front 2024-05-03T19:03:32.863665+0000 (oldest deadline 2024-05-03T19:03:57.563487+0000) 2024-05-03T19:04:11.819 INFO:journalctl@ceph.osd.6.smithi032.stdout:May 03 19:04:11 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-6[55806]: 2024-05-03T19:04:11.394+0000 7f7d9b701700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:40.464732+0000 front 2024-05-03T19:03:40.464684+0000 (oldest deadline 2024-05-03T19:04:05.164736+0000) 2024-05-03T19:04:11.998 INFO:journalctl@ceph.osd.7.smithi045.stdout:May 03 19:04:11 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-7[51710]: 2024-05-03T19:04:11.811+0000 7fe2c0194700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:32.077137+0000 front 2024-05-03T19:03:32.077303+0000 (oldest deadline 2024-05-03T19:03:53.777433+0000) 2024-05-03T19:04:11.999 INFO:journalctl@ceph.osd.7.smithi045.stdout:May 03 19:04:11 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-7[51710]: 2024-05-03T19:04:11.811+0000 7fe2c0194700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:39.579099+0000 front 2024-05-03T19:03:39.579074+0000 (oldest deadline 2024-05-03T19:04:05.479057+0000) 2024-05-03T19:04:11.999 INFO:journalctl@ceph.osd.7.smithi045.stdout:May 03 19:04:11 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-7[51710]: 2024-05-03T19:04:11.811+0000 7fe2c0194700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-05-03T19:03:45.479498+0000 front 2024-05-03T19:03:45.479594+0000 (oldest deadline 2024-05-03T19:04:10.779647+0000) 2024-05-03T19:04:11.999 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:11 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:11.618+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:25.050521+0000 front 2024-05-03T19:03:25.050536+0000 (oldest deadline 2024-05-03T19:03:49.150320+0000) 2024-05-03T19:04:11.999 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:11 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:11.618+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:29.150598+0000 front 2024-05-03T19:03:29.150627+0000 (oldest deadline 2024-05-03T19:03:55.050741+0000) 2024-05-03T19:04:11.999 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:11 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:11.618+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:37.951720+0000 front 2024-05-03T19:03:37.951864+0000 (oldest deadline 2024-05-03T19:04:02.651644+0000) 2024-05-03T19:04:12.301 INFO:journalctl@ceph.osd.8.smithi113.stdout:May 03 19:04:11 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-8[52667]: 2024-05-03T19:04:11.975+0000 7fb4a8e32700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:21.503361+0000 front 2024-05-03T19:03:21.503321+0000 (oldest deadline 2024-05-03T19:03:49.704026+0000) 2024-05-03T19:04:12.302 INFO:journalctl@ceph.osd.8.smithi113.stdout:May 03 19:04:11 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-8[52667]: 2024-05-03T19:04:11.975+0000 7fb4a8e32700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:30.805173+0000 front 2024-05-03T19:03:30.805157+0000 (oldest deadline 2024-05-03T19:03:53.605897+0000) 2024-05-03T19:04:12.302 INFO:journalctl@ceph.osd.8.smithi113.stdout:May 03 19:04:11 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-8[52667]: 2024-05-03T19:04:11.975+0000 7fb4a8e32700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:37.106650+0000 front 2024-05-03T19:03:37.106721+0000 (oldest deadline 2024-05-03T19:04:02.406804+0000) 2024-05-03T19:04:12.302 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:12 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:12.191+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:25.060144+0000 front 2024-05-03T19:03:25.060125+0000 (oldest deadline 2024-05-03T19:03:46.760321+0000) 2024-05-03T19:04:12.302 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:12 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:12.191+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:31.362284+0000 front 2024-05-03T19:03:31.362156+0000 (oldest deadline 2024-05-03T19:03:56.662340+0000) 2024-05-03T19:04:12.303 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:12 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:12.191+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:40.163402+0000 front 2024-05-03T19:03:40.163384+0000 (oldest deadline 2024-05-03T19:04:03.663534+0000) 2024-05-03T19:04:12.318 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:12 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:04:12.049+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:23.927793+0000 front 2024-05-03T19:03:23.927723+0000 (oldest deadline 2024-05-03T19:03:46.227921+0000) 2024-05-03T19:04:12.318 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:12 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:04:12.049+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:29.729118+0000 front 2024-05-03T19:03:29.729178+0000 (oldest deadline 2024-05-03T19:03:54.429041+0000) 2024-05-03T19:04:12.319 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:12 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:04:12.049+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:40.231110+0000 front 2024-05-03T19:03:40.230916+0000 (oldest deadline 2024-05-03T19:04:03.130962+0000) 2024-05-03T19:04:12.319 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:12 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:04:12.049+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-05-03T19:03:46.032125+0000 front 2024-05-03T19:03:46.032053+0000 (oldest deadline 2024-05-03T19:04:11.332075+0000) 2024-05-03T19:04:12.819 INFO:journalctl@ceph.osd.6.smithi032.stdout:May 03 19:04:12 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-6[55806]: 2024-05-03T19:04:12.367+0000 7f7d9b701700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:21.662275+0000 front 2024-05-03T19:03:21.662309+0000 (oldest deadline 2024-05-03T19:03:46.962364+0000) 2024-05-03T19:04:12.819 INFO:journalctl@ceph.osd.6.smithi032.stdout:May 03 19:04:12 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-6[55806]: 2024-05-03T19:04:12.367+0000 7f7d9b701700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:32.863448+0000 front 2024-05-03T19:03:32.863665+0000 (oldest deadline 2024-05-03T19:03:57.563487+0000) 2024-05-03T19:04:12.819 INFO:journalctl@ceph.osd.6.smithi032.stdout:May 03 19:04:12 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-6[55806]: 2024-05-03T19:04:12.367+0000 7f7d9b701700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:40.464732+0000 front 2024-05-03T19:03:40.464684+0000 (oldest deadline 2024-05-03T19:04:05.164736+0000) 2024-05-03T19:04:12.998 INFO:journalctl@ceph.osd.7.smithi045.stdout:May 03 19:04:12 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-7[51710]: 2024-05-03T19:04:12.785+0000 7fe2c0194700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:32.077137+0000 front 2024-05-03T19:03:32.077303+0000 (oldest deadline 2024-05-03T19:03:53.777433+0000) 2024-05-03T19:04:12.998 INFO:journalctl@ceph.osd.7.smithi045.stdout:May 03 19:04:12 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-7[51710]: 2024-05-03T19:04:12.785+0000 7fe2c0194700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:39.579099+0000 front 2024-05-03T19:03:39.579074+0000 (oldest deadline 2024-05-03T19:04:05.479057+0000) 2024-05-03T19:04:12.999 INFO:journalctl@ceph.osd.7.smithi045.stdout:May 03 19:04:12 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-7[51710]: 2024-05-03T19:04:12.785+0000 7fe2c0194700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-05-03T19:03:45.479498+0000 front 2024-05-03T19:03:45.479594+0000 (oldest deadline 2024-05-03T19:04:10.779647+0000) 2024-05-03T19:04:12.999 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:12 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:12.620+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:25.050521+0000 front 2024-05-03T19:03:25.050536+0000 (oldest deadline 2024-05-03T19:03:49.150320+0000) 2024-05-03T19:04:12.999 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:12 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:12.620+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:29.150598+0000 front 2024-05-03T19:03:29.150627+0000 (oldest deadline 2024-05-03T19:03:55.050741+0000) 2024-05-03T19:04:12.999 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:12 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:12.620+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:37.951720+0000 front 2024-05-03T19:03:37.951864+0000 (oldest deadline 2024-05-03T19:04:02.651644+0000) 2024-05-03T19:04:12.999 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:12 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:12.620+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-05-03T19:03:46.653109+0000 front 2024-05-03T19:03:46.653179+0000 (oldest deadline 2024-05-03T19:04:12.553001+0000) 2024-05-03T19:04:13.301 INFO:journalctl@ceph.osd.8.smithi113.stdout:May 03 19:04:13 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-8[52667]: 2024-05-03T19:04:13.010+0000 7fb4a8e32700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:21.503361+0000 front 2024-05-03T19:03:21.503321+0000 (oldest deadline 2024-05-03T19:03:49.704026+0000) 2024-05-03T19:04:13.302 INFO:journalctl@ceph.osd.8.smithi113.stdout:May 03 19:04:13 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-8[52667]: 2024-05-03T19:04:13.010+0000 7fb4a8e32700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:30.805173+0000 front 2024-05-03T19:03:30.805157+0000 (oldest deadline 2024-05-03T19:03:53.605897+0000) 2024-05-03T19:04:13.302 INFO:journalctl@ceph.osd.8.smithi113.stdout:May 03 19:04:13 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-8[52667]: 2024-05-03T19:04:13.010+0000 7fb4a8e32700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:37.106650+0000 front 2024-05-03T19:03:37.106721+0000 (oldest deadline 2024-05-03T19:04:02.406804+0000) 2024-05-03T19:04:13.302 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:13 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:13.212+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:25.060144+0000 front 2024-05-03T19:03:25.060125+0000 (oldest deadline 2024-05-03T19:03:46.760321+0000) 2024-05-03T19:04:13.302 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:13 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:13.212+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:31.362284+0000 front 2024-05-03T19:03:31.362156+0000 (oldest deadline 2024-05-03T19:03:56.662340+0000) 2024-05-03T19:04:13.302 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:13 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:13.212+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:40.163402+0000 front 2024-05-03T19:03:40.163384+0000 (oldest deadline 2024-05-03T19:04:03.663534+0000) 2024-05-03T19:04:13.318 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:13 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:04:13.058+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:23.927793+0000 front 2024-05-03T19:03:23.927723+0000 (oldest deadline 2024-05-03T19:03:46.227921+0000) 2024-05-03T19:04:13.319 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:13 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:04:13.058+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:29.729118+0000 front 2024-05-03T19:03:29.729178+0000 (oldest deadline 2024-05-03T19:03:54.429041+0000) 2024-05-03T19:04:13.319 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:13 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:04:13.058+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:40.231110+0000 front 2024-05-03T19:03:40.230916+0000 (oldest deadline 2024-05-03T19:04:03.130962+0000) 2024-05-03T19:04:13.319 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:13 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:04:13.058+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-05-03T19:03:46.032125+0000 front 2024-05-03T19:03:46.032053+0000 (oldest deadline 2024-05-03T19:04:11.332075+0000) 2024-05-03T19:04:13.818 INFO:journalctl@ceph.osd.6.smithi032.stdout:May 03 19:04:13 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-6[55806]: 2024-05-03T19:04:13.324+0000 7f7d9b701700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:21.662275+0000 front 2024-05-03T19:03:21.662309+0000 (oldest deadline 2024-05-03T19:03:46.962364+0000) 2024-05-03T19:04:13.819 INFO:journalctl@ceph.osd.6.smithi032.stdout:May 03 19:04:13 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-6[55806]: 2024-05-03T19:04:13.324+0000 7f7d9b701700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:32.863448+0000 front 2024-05-03T19:03:32.863665+0000 (oldest deadline 2024-05-03T19:03:57.563487+0000) 2024-05-03T19:04:13.819 INFO:journalctl@ceph.osd.6.smithi032.stdout:May 03 19:04:13 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-6[55806]: 2024-05-03T19:04:13.324+0000 7f7d9b701700 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:40.464732+0000 front 2024-05-03T19:03:40.464684+0000 (oldest deadline 2024-05-03T19:04:05.164736+0000) 2024-05-03T19:04:13.998 INFO:journalctl@ceph.osd.7.smithi045.stdout:May 03 19:04:13 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-7[51710]: 2024-05-03T19:04:13.739+0000 7fe2c0194700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:32.077137+0000 front 2024-05-03T19:03:32.077303+0000 (oldest deadline 2024-05-03T19:03:53.777433+0000) 2024-05-03T19:04:13.998 INFO:journalctl@ceph.osd.7.smithi045.stdout:May 03 19:04:13 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-7[51710]: 2024-05-03T19:04:13.739+0000 7fe2c0194700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:39.579099+0000 front 2024-05-03T19:03:39.579074+0000 (oldest deadline 2024-05-03T19:04:05.479057+0000) 2024-05-03T19:04:13.998 INFO:journalctl@ceph.osd.7.smithi045.stdout:May 03 19:04:13 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-7[51710]: 2024-05-03T19:04:13.739+0000 7fe2c0194700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-05-03T19:03:45.479498+0000 front 2024-05-03T19:03:45.479594+0000 (oldest deadline 2024-05-03T19:04:10.779647+0000) 2024-05-03T19:04:13.999 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:13 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:13.648+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:25.050521+0000 front 2024-05-03T19:03:25.050536+0000 (oldest deadline 2024-05-03T19:03:49.150320+0000) 2024-05-03T19:04:13.999 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:13 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:13.648+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:29.150598+0000 front 2024-05-03T19:03:29.150627+0000 (oldest deadline 2024-05-03T19:03:55.050741+0000) 2024-05-03T19:04:13.999 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:13 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:13.648+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:37.951720+0000 front 2024-05-03T19:03:37.951864+0000 (oldest deadline 2024-05-03T19:04:02.651644+0000) 2024-05-03T19:04:13.999 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:13 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:13.648+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-05-03T19:03:46.653109+0000 front 2024-05-03T19:03:46.653179+0000 (oldest deadline 2024-05-03T19:04:12.553001+0000) 2024-05-03T19:04:14.192 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:14 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:04:14.094+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:23.927793+0000 front 2024-05-03T19:03:23.927723+0000 (oldest deadline 2024-05-03T19:03:46.227921+0000) 2024-05-03T19:04:14.193 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:14 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:04:14.094+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:29.729118+0000 front 2024-05-03T19:03:29.729178+0000 (oldest deadline 2024-05-03T19:03:54.429041+0000) 2024-05-03T19:04:14.193 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:14 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:04:14.094+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:40.231110+0000 front 2024-05-03T19:03:40.230916+0000 (oldest deadline 2024-05-03T19:04:03.130962+0000) 2024-05-03T19:04:14.193 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:14 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:04:14.094+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-05-03T19:03:46.032125+0000 front 2024-05-03T19:03:46.032053+0000 (oldest deadline 2024-05-03T19:04:11.332075+0000) 2024-05-03T19:04:14.193 INFO:journalctl@ceph.osd.6.smithi032.stdout:May 03 19:04:13 smithi032 podman[87976]: 2024-05-03 19:04:13.920761665 +0000 UTC m=+5.666913014 container died 56afac64d3ede1848f9546bc751dae4263a8b992d458e1c3fabfee2ac57b6494 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-6, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240430, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.schema-version=1.0) 2024-05-03T19:04:14.243 INFO:journalctl@ceph.osd.8.smithi113.stdout:May 03 19:04:13 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-8[52667]: 2024-05-03T19:04:13.962+0000 7fb4a8e32700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:21.503361+0000 front 2024-05-03T19:03:21.503321+0000 (oldest deadline 2024-05-03T19:03:49.704026+0000) 2024-05-03T19:04:14.243 INFO:journalctl@ceph.osd.8.smithi113.stdout:May 03 19:04:13 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-8[52667]: 2024-05-03T19:04:13.962+0000 7fb4a8e32700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:30.805173+0000 front 2024-05-03T19:03:30.805157+0000 (oldest deadline 2024-05-03T19:03:53.605897+0000) 2024-05-03T19:04:14.243 INFO:journalctl@ceph.osd.8.smithi113.stdout:May 03 19:04:13 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-8[52667]: 2024-05-03T19:04:13.962+0000 7fb4a8e32700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:37.106650+0000 front 2024-05-03T19:03:37.106721+0000 (oldest deadline 2024-05-03T19:04:02.406804+0000) 2024-05-03T19:04:14.542 INFO:journalctl@ceph.osd.6.smithi032.stdout:May 03 19:04:14 smithi032 podman[87976]: 2024-05-03 19:04:14.282506407 +0000 UTC m=+6.028657752 container cleanup 56afac64d3ede1848f9546bc751dae4263a8b992d458e1c3fabfee2ac57b6494 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-6, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240430, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-05-03T19:04:14.542 INFO:journalctl@ceph.osd.6.smithi032.stdout:May 03 19:04:14 smithi032 bash[87976]: ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-6 2024-05-03T19:04:14.551 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:14 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:14.242+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:25.060144+0000 front 2024-05-03T19:03:25.060125+0000 (oldest deadline 2024-05-03T19:03:46.760321+0000) 2024-05-03T19:04:14.551 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:14 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:14.242+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:31.362284+0000 front 2024-05-03T19:03:31.362156+0000 (oldest deadline 2024-05-03T19:03:56.662340+0000) 2024-05-03T19:04:14.551 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:14 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:14.242+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:40.163402+0000 front 2024-05-03T19:03:40.163384+0000 (oldest deadline 2024-05-03T19:04:03.663534+0000) 2024-05-03T19:04:14.818 INFO:journalctl@ceph.osd.6.smithi032.stdout:May 03 19:04:14 smithi032 podman[87994]: 2024-05-03 19:04:14.541684961 +0000 UTC m=+0.619627910 container remove 56afac64d3ede1848f9546bc751dae4263a8b992d458e1c3fabfee2ac57b6494 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-6, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240430, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-05-03T19:04:14.998 INFO:journalctl@ceph.osd.7.smithi045.stdout:May 03 19:04:14 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-7[51710]: 2024-05-03T19:04:14.714+0000 7fe2c0194700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:32.077137+0000 front 2024-05-03T19:03:32.077303+0000 (oldest deadline 2024-05-03T19:03:53.777433+0000) 2024-05-03T19:04:14.998 INFO:journalctl@ceph.osd.7.smithi045.stdout:May 03 19:04:14 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-7[51710]: 2024-05-03T19:04:14.714+0000 7fe2c0194700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:39.579099+0000 front 2024-05-03T19:03:39.579074+0000 (oldest deadline 2024-05-03T19:04:05.479057+0000) 2024-05-03T19:04:14.998 INFO:journalctl@ceph.osd.7.smithi045.stdout:May 03 19:04:14 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-7[51710]: 2024-05-03T19:04:14.714+0000 7fe2c0194700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-05-03T19:03:45.479498+0000 front 2024-05-03T19:03:45.479594+0000 (oldest deadline 2024-05-03T19:04:10.779647+0000) 2024-05-03T19:04:14.999 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:14 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:14.661+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:25.050521+0000 front 2024-05-03T19:03:25.050536+0000 (oldest deadline 2024-05-03T19:03:49.150320+0000) 2024-05-03T19:04:14.999 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:14 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:14.661+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:29.150598+0000 front 2024-05-03T19:03:29.150627+0000 (oldest deadline 2024-05-03T19:03:55.050741+0000) 2024-05-03T19:04:14.999 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:14 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:14.661+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:37.951720+0000 front 2024-05-03T19:03:37.951864+0000 (oldest deadline 2024-05-03T19:04:02.651644+0000) 2024-05-03T19:04:14.999 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:14 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:14.661+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-05-03T19:03:46.653109+0000 front 2024-05-03T19:03:46.653179+0000 (oldest deadline 2024-05-03T19:04:12.553001+0000) 2024-05-03T19:04:15.170 INFO:journalctl@ceph.osd.6.smithi032.stdout:May 03 19:04:14 smithi032 podman[88081]: 2024-05-03 19:04:14.771813897 +0000 UTC m=+0.018556223 image pull ee4d846202496ba6096593e8e8becc0700a5eb7ebd95536031187f5f337d4012 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd 2024-05-03T19:04:15.171 INFO:journalctl@ceph.osd.6.smithi032.stdout:May 03 19:04:14 smithi032 podman[88081]: 2024-05-03 19:04:14.875912761 +0000 UTC m=+0.122655098 container create f674459671ce3d7a131a08cfdc4920a9b4273da34a6a81d2a08eab93e0cd2894 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-6-deactivate, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240430, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe) 2024-05-03T19:04:15.171 INFO:journalctl@ceph.osd.6.smithi032.stdout:May 03 19:04:15 smithi032 podman[88081]: 2024-05-03 19:04:15.094863507 +0000 UTC m=+0.341605805 container init f674459671ce3d7a131a08cfdc4920a9b4273da34a6a81d2a08eab93e0cd2894 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-6-deactivate, ceph=True, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240430, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-05-03T19:04:15.171 INFO:journalctl@ceph.osd.6.smithi032.stdout:May 03 19:04:15 smithi032 podman[88081]: 2024-05-03 19:04:15.098045733 +0000 UTC m=+0.344788034 container start f674459671ce3d7a131a08cfdc4920a9b4273da34a6a81d2a08eab93e0cd2894 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-6-deactivate, org.label-schema.build-date=20240430, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-05-03T19:04:15.171 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:15 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:04:15.055+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:23.927793+0000 front 2024-05-03T19:03:23.927723+0000 (oldest deadline 2024-05-03T19:03:46.227921+0000) 2024-05-03T19:04:15.172 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:15 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:04:15.055+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:29.729118+0000 front 2024-05-03T19:03:29.729178+0000 (oldest deadline 2024-05-03T19:03:54.429041+0000) 2024-05-03T19:04:15.172 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:15 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:04:15.055+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:40.231110+0000 front 2024-05-03T19:03:40.230916+0000 (oldest deadline 2024-05-03T19:04:03.130962+0000) 2024-05-03T19:04:15.172 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:15 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:04:15.055+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-05-03T19:03:46.032125+0000 front 2024-05-03T19:03:46.032053+0000 (oldest deadline 2024-05-03T19:04:11.332075+0000) 2024-05-03T19:04:15.270 INFO:journalctl@ceph.osd.8.smithi113.stdout:May 03 19:04:14 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-8[52667]: 2024-05-03T19:04:14.931+0000 7fb4a8e32700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:21.503361+0000 front 2024-05-03T19:03:21.503321+0000 (oldest deadline 2024-05-03T19:03:49.704026+0000) 2024-05-03T19:04:15.270 INFO:journalctl@ceph.osd.8.smithi113.stdout:May 03 19:04:14 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-8[52667]: 2024-05-03T19:04:14.931+0000 7fb4a8e32700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:30.805173+0000 front 2024-05-03T19:03:30.805157+0000 (oldest deadline 2024-05-03T19:03:53.605897+0000) 2024-05-03T19:04:15.271 INFO:journalctl@ceph.osd.8.smithi113.stdout:May 03 19:04:14 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-8[52667]: 2024-05-03T19:04:14.931+0000 7fb4a8e32700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:37.106650+0000 front 2024-05-03T19:03:37.106721+0000 (oldest deadline 2024-05-03T19:04:02.406804+0000) 2024-05-03T19:04:15.551 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:15 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:15.269+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:25.060144+0000 front 2024-05-03T19:03:25.060125+0000 (oldest deadline 2024-05-03T19:03:46.760321+0000) 2024-05-03T19:04:15.552 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:15 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:15.269+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:31.362284+0000 front 2024-05-03T19:03:31.362156+0000 (oldest deadline 2024-05-03T19:03:56.662340+0000) 2024-05-03T19:04:15.552 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:15 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:15.269+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:40.163402+0000 front 2024-05-03T19:03:40.163384+0000 (oldest deadline 2024-05-03T19:04:03.663534+0000) 2024-05-03T19:04:15.569 INFO:journalctl@ceph.osd.6.smithi032.stdout:May 03 19:04:15 smithi032 podman[88081]: 2024-05-03 19:04:15.169542447 +0000 UTC m=+0.416284750 container attach f674459671ce3d7a131a08cfdc4920a9b4273da34a6a81d2a08eab93e0cd2894 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-6-deactivate, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240430, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-03T19:04:15.569 INFO:journalctl@ceph.osd.6.smithi032.stdout:May 03 19:04:15 smithi032 podman[88081]: 2024-05-03 19:04:15.409588292 +0000 UTC m=+0.656330602 container died f674459671ce3d7a131a08cfdc4920a9b4273da34a6a81d2a08eab93e0cd2894 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-6-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240430, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-05-03T19:04:15.966 INFO:journalctl@ceph.osd.6.smithi032.stdout:May 03 19:04:15 smithi032 podman[88107]: 2024-05-03 19:04:15.939551857 +0000 UTC m=+0.523905200 container remove f674459671ce3d7a131a08cfdc4920a9b4273da34a6a81d2a08eab93e0cd2894 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-6-deactivate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240430) 2024-05-03T19:04:15.972 DEBUG:teuthology.orchestra.run.smithi032:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f500455e-097b-11ef-bc97-c7b262605968@osd.6.service' 2024-05-03T19:04:15.998 INFO:journalctl@ceph.osd.7.smithi045.stdout:May 03 19:04:15 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-7[51710]: 2024-05-03T19:04:15.730+0000 7fe2c0194700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:32.077137+0000 front 2024-05-03T19:03:32.077303+0000 (oldest deadline 2024-05-03T19:03:53.777433+0000) 2024-05-03T19:04:15.998 INFO:journalctl@ceph.osd.7.smithi045.stdout:May 03 19:04:15 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-7[51710]: 2024-05-03T19:04:15.730+0000 7fe2c0194700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:39.579099+0000 front 2024-05-03T19:03:39.579074+0000 (oldest deadline 2024-05-03T19:04:05.479057+0000) 2024-05-03T19:04:15.998 INFO:journalctl@ceph.osd.7.smithi045.stdout:May 03 19:04:15 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-7[51710]: 2024-05-03T19:04:15.730+0000 7fe2c0194700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-05-03T19:03:45.479498+0000 front 2024-05-03T19:03:45.479594+0000 (oldest deadline 2024-05-03T19:04:10.779647+0000) 2024-05-03T19:04:15.999 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:15 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:15.695+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:25.050521+0000 front 2024-05-03T19:03:25.050536+0000 (oldest deadline 2024-05-03T19:03:49.150320+0000) 2024-05-03T19:04:15.999 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:15 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:15.695+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:29.150598+0000 front 2024-05-03T19:03:29.150627+0000 (oldest deadline 2024-05-03T19:03:55.050741+0000) 2024-05-03T19:04:15.999 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:15 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:15.695+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:37.951720+0000 front 2024-05-03T19:03:37.951864+0000 (oldest deadline 2024-05-03T19:04:02.651644+0000) 2024-05-03T19:04:15.999 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:15 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:15.695+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-05-03T19:03:46.653109+0000 front 2024-05-03T19:03:46.653179+0000 (oldest deadline 2024-05-03T19:04:12.553001+0000) 2024-05-03T19:04:16.012 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-03T19:04:16.013 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-05-03T19:04:16.013 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-05-03T19:04:16.013 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ceph-f500455e-097b-11ef-bc97-c7b262605968@osd.7 2024-05-03T19:04:16.281 INFO:journalctl@ceph.osd.8.smithi113.stdout:May 03 19:04:15 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-8[52667]: 2024-05-03T19:04:15.910+0000 7fb4a8e32700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:21.503361+0000 front 2024-05-03T19:03:21.503321+0000 (oldest deadline 2024-05-03T19:03:49.704026+0000) 2024-05-03T19:04:16.282 INFO:journalctl@ceph.osd.8.smithi113.stdout:May 03 19:04:15 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-8[52667]: 2024-05-03T19:04:15.910+0000 7fb4a8e32700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:30.805173+0000 front 2024-05-03T19:03:30.805157+0000 (oldest deadline 2024-05-03T19:03:53.605897+0000) 2024-05-03T19:04:16.282 INFO:journalctl@ceph.osd.8.smithi113.stdout:May 03 19:04:15 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-8[52667]: 2024-05-03T19:04:15.910+0000 7fb4a8e32700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:37.106650+0000 front 2024-05-03T19:03:37.106721+0000 (oldest deadline 2024-05-03T19:04:02.406804+0000) 2024-05-03T19:04:16.318 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:16 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:04:16.088+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:23.927793+0000 front 2024-05-03T19:03:23.927723+0000 (oldest deadline 2024-05-03T19:03:46.227921+0000) 2024-05-03T19:04:16.319 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:16 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:04:16.088+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:29.729118+0000 front 2024-05-03T19:03:29.729178+0000 (oldest deadline 2024-05-03T19:03:54.429041+0000) 2024-05-03T19:04:16.319 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:16 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:04:16.088+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:40.231110+0000 front 2024-05-03T19:03:40.230916+0000 (oldest deadline 2024-05-03T19:04:03.130962+0000) 2024-05-03T19:04:16.319 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:16 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:04:16.088+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-05-03T19:03:46.032125+0000 front 2024-05-03T19:03:46.032053+0000 (oldest deadline 2024-05-03T19:04:11.332075+0000) 2024-05-03T19:04:16.322 INFO:journalctl@ceph.osd.7.smithi045.stdout:May 03 19:04:16 smithi045 systemd[1]: Stopping Ceph osd.7 for f500455e-097b-11ef-bc97-c7b262605968... 2024-05-03T19:04:16.551 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:16 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:16.280+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:25.060144+0000 front 2024-05-03T19:03:25.060125+0000 (oldest deadline 2024-05-03T19:03:46.760321+0000) 2024-05-03T19:04:16.551 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:16 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:16.280+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:31.362284+0000 front 2024-05-03T19:03:31.362156+0000 (oldest deadline 2024-05-03T19:03:56.662340+0000) 2024-05-03T19:04:16.552 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:16 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:16.280+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:40.163402+0000 front 2024-05-03T19:03:40.163384+0000 (oldest deadline 2024-05-03T19:04:03.663534+0000) 2024-05-03T19:04:16.694 INFO:journalctl@ceph.osd.7.smithi045.stdout:May 03 19:04:16 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-7[51710]: 2024-05-03T19:04:16.320+0000 7fe2c7ba8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-03T19:04:16.694 INFO:journalctl@ceph.osd.7.smithi045.stdout:May 03 19:04:16 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-7[51710]: 2024-05-03T19:04:16.320+0000 7fe2c7ba8700 -1 osd.7 85 *** Got signal Terminated *** 2024-05-03T19:04:16.694 INFO:journalctl@ceph.osd.7.smithi045.stdout:May 03 19:04:16 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-7[51710]: 2024-05-03T19:04:16.320+0000 7fe2c7ba8700 -1 osd.7 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-03T19:04:16.998 INFO:journalctl@ceph.osd.7.smithi045.stdout:May 03 19:04:16 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-7[51710]: 2024-05-03T19:04:16.746+0000 7fe2c0194700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:32.077137+0000 front 2024-05-03T19:03:32.077303+0000 (oldest deadline 2024-05-03T19:03:53.777433+0000) 2024-05-03T19:04:16.999 INFO:journalctl@ceph.osd.7.smithi045.stdout:May 03 19:04:16 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-7[51710]: 2024-05-03T19:04:16.746+0000 7fe2c0194700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:39.579099+0000 front 2024-05-03T19:03:39.579074+0000 (oldest deadline 2024-05-03T19:04:05.479057+0000) 2024-05-03T19:04:16.999 INFO:journalctl@ceph.osd.7.smithi045.stdout:May 03 19:04:16 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-7[51710]: 2024-05-03T19:04:16.746+0000 7fe2c0194700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-05-03T19:03:45.479498+0000 front 2024-05-03T19:03:45.479594+0000 (oldest deadline 2024-05-03T19:04:10.779647+0000) 2024-05-03T19:04:16.999 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:16 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:16.692+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:25.050521+0000 front 2024-05-03T19:03:25.050536+0000 (oldest deadline 2024-05-03T19:03:49.150320+0000) 2024-05-03T19:04:16.999 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:16 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:16.692+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:29.150598+0000 front 2024-05-03T19:03:29.150627+0000 (oldest deadline 2024-05-03T19:03:55.050741+0000) 2024-05-03T19:04:16.999 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:16 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:16.692+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:37.951720+0000 front 2024-05-03T19:03:37.951864+0000 (oldest deadline 2024-05-03T19:04:02.651644+0000) 2024-05-03T19:04:16.999 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:16 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:16.692+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-05-03T19:03:46.653109+0000 front 2024-05-03T19:03:46.653179+0000 (oldest deadline 2024-05-03T19:04:12.553001+0000) 2024-05-03T19:04:17.301 INFO:journalctl@ceph.osd.8.smithi113.stdout:May 03 19:04:16 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-8[52667]: 2024-05-03T19:04:16.898+0000 7fb4a8e32700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:21.503361+0000 front 2024-05-03T19:03:21.503321+0000 (oldest deadline 2024-05-03T19:03:49.704026+0000) 2024-05-03T19:04:17.301 INFO:journalctl@ceph.osd.8.smithi113.stdout:May 03 19:04:16 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-8[52667]: 2024-05-03T19:04:16.898+0000 7fb4a8e32700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:30.805173+0000 front 2024-05-03T19:03:30.805157+0000 (oldest deadline 2024-05-03T19:03:53.605897+0000) 2024-05-03T19:04:17.302 INFO:journalctl@ceph.osd.8.smithi113.stdout:May 03 19:04:16 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-8[52667]: 2024-05-03T19:04:16.898+0000 7fb4a8e32700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:37.106650+0000 front 2024-05-03T19:03:37.106721+0000 (oldest deadline 2024-05-03T19:04:02.406804+0000) 2024-05-03T19:04:17.569 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:17 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:04:17.081+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:23.927793+0000 front 2024-05-03T19:03:23.927723+0000 (oldest deadline 2024-05-03T19:03:46.227921+0000) 2024-05-03T19:04:17.569 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:17 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:04:17.081+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:29.729118+0000 front 2024-05-03T19:03:29.729178+0000 (oldest deadline 2024-05-03T19:03:54.429041+0000) 2024-05-03T19:04:17.569 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:17 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:04:17.081+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:40.231110+0000 front 2024-05-03T19:03:40.230916+0000 (oldest deadline 2024-05-03T19:04:03.130962+0000) 2024-05-03T19:04:17.569 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:17 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:04:17.081+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-05-03T19:03:46.032125+0000 front 2024-05-03T19:03:46.032053+0000 (oldest deadline 2024-05-03T19:04:11.332075+0000) 2024-05-03T19:04:17.801 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:17 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:17.309+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:25.060144+0000 front 2024-05-03T19:03:25.060125+0000 (oldest deadline 2024-05-03T19:03:46.760321+0000) 2024-05-03T19:04:17.802 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:17 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:17.309+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:31.362284+0000 front 2024-05-03T19:03:31.362156+0000 (oldest deadline 2024-05-03T19:03:56.662340+0000) 2024-05-03T19:04:17.802 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:17 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:17.309+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:40.163402+0000 front 2024-05-03T19:03:40.163384+0000 (oldest deadline 2024-05-03T19:04:03.663534+0000) 2024-05-03T19:04:17.998 INFO:journalctl@ceph.osd.7.smithi045.stdout:May 03 19:04:17 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-7[51710]: 2024-05-03T19:04:17.732+0000 7fe2c0194700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:32.077137+0000 front 2024-05-03T19:03:32.077303+0000 (oldest deadline 2024-05-03T19:03:53.777433+0000) 2024-05-03T19:04:17.998 INFO:journalctl@ceph.osd.7.smithi045.stdout:May 03 19:04:17 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-7[51710]: 2024-05-03T19:04:17.732+0000 7fe2c0194700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:39.579099+0000 front 2024-05-03T19:03:39.579074+0000 (oldest deadline 2024-05-03T19:04:05.479057+0000) 2024-05-03T19:04:17.999 INFO:journalctl@ceph.osd.7.smithi045.stdout:May 03 19:04:17 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-7[51710]: 2024-05-03T19:04:17.732+0000 7fe2c0194700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-05-03T19:03:45.479498+0000 front 2024-05-03T19:03:45.479594+0000 (oldest deadline 2024-05-03T19:04:10.779647+0000) 2024-05-03T19:04:17.999 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:17 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:17.674+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:25.050521+0000 front 2024-05-03T19:03:25.050536+0000 (oldest deadline 2024-05-03T19:03:49.150320+0000) 2024-05-03T19:04:17.999 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:17 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:17.674+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:29.150598+0000 front 2024-05-03T19:03:29.150627+0000 (oldest deadline 2024-05-03T19:03:55.050741+0000) 2024-05-03T19:04:17.999 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:17 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:17.674+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:37.951720+0000 front 2024-05-03T19:03:37.951864+0000 (oldest deadline 2024-05-03T19:04:02.651644+0000) 2024-05-03T19:04:17.999 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:17 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:17.674+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-05-03T19:03:46.653109+0000 front 2024-05-03T19:03:46.653179+0000 (oldest deadline 2024-05-03T19:04:12.553001+0000) 2024-05-03T19:04:18.301 INFO:journalctl@ceph.osd.8.smithi113.stdout:May 03 19:04:17 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-8[52667]: 2024-05-03T19:04:17.894+0000 7fb4a8e32700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:21.503361+0000 front 2024-05-03T19:03:21.503321+0000 (oldest deadline 2024-05-03T19:03:49.704026+0000) 2024-05-03T19:04:18.301 INFO:journalctl@ceph.osd.8.smithi113.stdout:May 03 19:04:17 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-8[52667]: 2024-05-03T19:04:17.894+0000 7fb4a8e32700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:30.805173+0000 front 2024-05-03T19:03:30.805157+0000 (oldest deadline 2024-05-03T19:03:53.605897+0000) 2024-05-03T19:04:18.302 INFO:journalctl@ceph.osd.8.smithi113.stdout:May 03 19:04:17 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-8[52667]: 2024-05-03T19:04:17.894+0000 7fb4a8e32700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:37.106650+0000 front 2024-05-03T19:03:37.106721+0000 (oldest deadline 2024-05-03T19:04:02.406804+0000) 2024-05-03T19:04:18.568 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:18 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:04:18.069+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:23.927793+0000 front 2024-05-03T19:03:23.927723+0000 (oldest deadline 2024-05-03T19:03:46.227921+0000) 2024-05-03T19:04:18.569 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:18 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:04:18.069+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:29.729118+0000 front 2024-05-03T19:03:29.729178+0000 (oldest deadline 2024-05-03T19:03:54.429041+0000) 2024-05-03T19:04:18.569 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:18 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:04:18.069+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:40.231110+0000 front 2024-05-03T19:03:40.230916+0000 (oldest deadline 2024-05-03T19:04:03.130962+0000) 2024-05-03T19:04:18.569 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:18 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:04:18.069+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-05-03T19:03:46.032125+0000 front 2024-05-03T19:03:46.032053+0000 (oldest deadline 2024-05-03T19:04:11.332075+0000) 2024-05-03T19:04:18.801 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:18 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:18.343+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:25.060144+0000 front 2024-05-03T19:03:25.060125+0000 (oldest deadline 2024-05-03T19:03:46.760321+0000) 2024-05-03T19:04:18.802 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:18 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:18.343+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:31.362284+0000 front 2024-05-03T19:03:31.362156+0000 (oldest deadline 2024-05-03T19:03:56.662340+0000) 2024-05-03T19:04:18.802 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:18 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:18.343+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:40.163402+0000 front 2024-05-03T19:03:40.163384+0000 (oldest deadline 2024-05-03T19:04:03.663534+0000) 2024-05-03T19:04:18.998 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:18 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:18.708+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:25.050521+0000 front 2024-05-03T19:03:25.050536+0000 (oldest deadline 2024-05-03T19:03:49.150320+0000) 2024-05-03T19:04:18.998 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:18 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:18.708+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:29.150598+0000 front 2024-05-03T19:03:29.150627+0000 (oldest deadline 2024-05-03T19:03:55.050741+0000) 2024-05-03T19:04:18.998 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:18 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:18.708+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:37.951720+0000 front 2024-05-03T19:03:37.951864+0000 (oldest deadline 2024-05-03T19:04:02.651644+0000) 2024-05-03T19:04:18.999 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:18 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:18.708+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-05-03T19:03:46.653109+0000 front 2024-05-03T19:03:46.653179+0000 (oldest deadline 2024-05-03T19:04:12.553001+0000) 2024-05-03T19:04:18.999 INFO:journalctl@ceph.osd.7.smithi045.stdout:May 03 19:04:18 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-7[51710]: 2024-05-03T19:04:18.701+0000 7fe2c0194700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:32.077137+0000 front 2024-05-03T19:03:32.077303+0000 (oldest deadline 2024-05-03T19:03:53.777433+0000) 2024-05-03T19:04:18.999 INFO:journalctl@ceph.osd.7.smithi045.stdout:May 03 19:04:18 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-7[51710]: 2024-05-03T19:04:18.701+0000 7fe2c0194700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:39.579099+0000 front 2024-05-03T19:03:39.579074+0000 (oldest deadline 2024-05-03T19:04:05.479057+0000) 2024-05-03T19:04:18.999 INFO:journalctl@ceph.osd.7.smithi045.stdout:May 03 19:04:18 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-7[51710]: 2024-05-03T19:04:18.701+0000 7fe2c0194700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-05-03T19:03:45.479498+0000 front 2024-05-03T19:03:45.479594+0000 (oldest deadline 2024-05-03T19:04:10.779647+0000) 2024-05-03T19:04:19.301 INFO:journalctl@ceph.osd.8.smithi113.stdout:May 03 19:04:18 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-8[52667]: 2024-05-03T19:04:18.867+0000 7fb4a8e32700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:21.503361+0000 front 2024-05-03T19:03:21.503321+0000 (oldest deadline 2024-05-03T19:03:49.704026+0000) 2024-05-03T19:04:19.302 INFO:journalctl@ceph.osd.8.smithi113.stdout:May 03 19:04:18 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-8[52667]: 2024-05-03T19:04:18.867+0000 7fb4a8e32700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:30.805173+0000 front 2024-05-03T19:03:30.805157+0000 (oldest deadline 2024-05-03T19:03:53.605897+0000) 2024-05-03T19:04:19.302 INFO:journalctl@ceph.osd.8.smithi113.stdout:May 03 19:04:18 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-8[52667]: 2024-05-03T19:04:18.867+0000 7fb4a8e32700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:37.106650+0000 front 2024-05-03T19:03:37.106721+0000 (oldest deadline 2024-05-03T19:04:02.406804+0000) 2024-05-03T19:04:19.302 INFO:journalctl@ceph.osd.8.smithi113.stdout:May 03 19:04:18 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-8[52667]: 2024-05-03T19:04:18.867+0000 7fb4a8e32700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.45:6812 osd.4 since back 2024-05-03T19:03:57.009866+0000 front 2024-05-03T19:03:57.009783+0000 (oldest deadline 2024-05-03T19:04:18.709619+0000) 2024-05-03T19:04:19.318 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:19 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:04:19.024+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:23.927793+0000 front 2024-05-03T19:03:23.927723+0000 (oldest deadline 2024-05-03T19:03:46.227921+0000) 2024-05-03T19:04:19.318 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:19 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:04:19.024+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:29.729118+0000 front 2024-05-03T19:03:29.729178+0000 (oldest deadline 2024-05-03T19:03:54.429041+0000) 2024-05-03T19:04:19.319 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:19 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:04:19.024+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:40.231110+0000 front 2024-05-03T19:03:40.230916+0000 (oldest deadline 2024-05-03T19:04:03.130962+0000) 2024-05-03T19:04:19.319 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:19 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:04:19.024+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-05-03T19:03:46.032125+0000 front 2024-05-03T19:03:46.032053+0000 (oldest deadline 2024-05-03T19:04:11.332075+0000) 2024-05-03T19:04:19.801 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:19 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:19.365+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:25.060144+0000 front 2024-05-03T19:03:25.060125+0000 (oldest deadline 2024-05-03T19:03:46.760321+0000) 2024-05-03T19:04:19.802 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:19 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:19.365+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:31.362284+0000 front 2024-05-03T19:03:31.362156+0000 (oldest deadline 2024-05-03T19:03:56.662340+0000) 2024-05-03T19:04:19.802 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:19 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:19.365+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:40.163402+0000 front 2024-05-03T19:03:40.163384+0000 (oldest deadline 2024-05-03T19:04:03.663534+0000) 2024-05-03T19:04:19.998 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:19 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:19.674+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:25.050521+0000 front 2024-05-03T19:03:25.050536+0000 (oldest deadline 2024-05-03T19:03:49.150320+0000) 2024-05-03T19:04:19.998 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:19 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:19.674+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:29.150598+0000 front 2024-05-03T19:03:29.150627+0000 (oldest deadline 2024-05-03T19:03:55.050741+0000) 2024-05-03T19:04:19.998 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:19 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:19.674+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:37.951720+0000 front 2024-05-03T19:03:37.951864+0000 (oldest deadline 2024-05-03T19:04:02.651644+0000) 2024-05-03T19:04:19.998 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:19 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:19.674+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-05-03T19:03:46.653109+0000 front 2024-05-03T19:03:46.653179+0000 (oldest deadline 2024-05-03T19:04:12.553001+0000) 2024-05-03T19:04:19.999 INFO:journalctl@ceph.osd.7.smithi045.stdout:May 03 19:04:19 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-7[51710]: 2024-05-03T19:04:19.696+0000 7fe2c0194700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:32.077137+0000 front 2024-05-03T19:03:32.077303+0000 (oldest deadline 2024-05-03T19:03:53.777433+0000) 2024-05-03T19:04:19.999 INFO:journalctl@ceph.osd.7.smithi045.stdout:May 03 19:04:19 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-7[51710]: 2024-05-03T19:04:19.696+0000 7fe2c0194700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:39.579099+0000 front 2024-05-03T19:03:39.579074+0000 (oldest deadline 2024-05-03T19:04:05.479057+0000) 2024-05-03T19:04:19.999 INFO:journalctl@ceph.osd.7.smithi045.stdout:May 03 19:04:19 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-7[51710]: 2024-05-03T19:04:19.696+0000 7fe2c0194700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-05-03T19:03:45.479498+0000 front 2024-05-03T19:03:45.479594+0000 (oldest deadline 2024-05-03T19:04:10.779647+0000) 2024-05-03T19:04:19.999 INFO:journalctl@ceph.osd.7.smithi045.stdout:May 03 19:04:19 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-7[51710]: 2024-05-03T19:04:19.696+0000 7fe2c0194700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.45:6812 osd.4 since back 2024-05-03T19:03:54.280511+0000 front 2024-05-03T19:03:54.280483+0000 (oldest deadline 2024-05-03T19:04:18.980674+0000) 2024-05-03T19:04:20.301 INFO:journalctl@ceph.osd.8.smithi113.stdout:May 03 19:04:19 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-8[52667]: 2024-05-03T19:04:19.868+0000 7fb4a8e32700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:21.503361+0000 front 2024-05-03T19:03:21.503321+0000 (oldest deadline 2024-05-03T19:03:49.704026+0000) 2024-05-03T19:04:20.302 INFO:journalctl@ceph.osd.8.smithi113.stdout:May 03 19:04:19 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-8[52667]: 2024-05-03T19:04:19.868+0000 7fb4a8e32700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:30.805173+0000 front 2024-05-03T19:03:30.805157+0000 (oldest deadline 2024-05-03T19:03:53.605897+0000) 2024-05-03T19:04:20.302 INFO:journalctl@ceph.osd.8.smithi113.stdout:May 03 19:04:19 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-8[52667]: 2024-05-03T19:04:19.868+0000 7fb4a8e32700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:37.106650+0000 front 2024-05-03T19:03:37.106721+0000 (oldest deadline 2024-05-03T19:04:02.406804+0000) 2024-05-03T19:04:20.302 INFO:journalctl@ceph.osd.8.smithi113.stdout:May 03 19:04:19 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-8[52667]: 2024-05-03T19:04:19.868+0000 7fb4a8e32700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.45:6812 osd.4 since back 2024-05-03T19:03:57.009866+0000 front 2024-05-03T19:03:57.009783+0000 (oldest deadline 2024-05-03T19:04:18.709619+0000) 2024-05-03T19:04:20.568 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:20 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:04:20.070+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:23.927793+0000 front 2024-05-03T19:03:23.927723+0000 (oldest deadline 2024-05-03T19:03:46.227921+0000) 2024-05-03T19:04:20.569 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:20 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:04:20.070+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:29.729118+0000 front 2024-05-03T19:03:29.729178+0000 (oldest deadline 2024-05-03T19:03:54.429041+0000) 2024-05-03T19:04:20.569 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:20 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:04:20.070+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:40.231110+0000 front 2024-05-03T19:03:40.230916+0000 (oldest deadline 2024-05-03T19:04:03.130962+0000) 2024-05-03T19:04:20.569 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:20 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:04:20.070+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-05-03T19:03:46.032125+0000 front 2024-05-03T19:03:46.032053+0000 (oldest deadline 2024-05-03T19:04:11.332075+0000) 2024-05-03T19:04:20.801 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:20 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:20.367+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:25.060144+0000 front 2024-05-03T19:03:25.060125+0000 (oldest deadline 2024-05-03T19:03:46.760321+0000) 2024-05-03T19:04:20.802 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:20 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:20.367+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:31.362284+0000 front 2024-05-03T19:03:31.362156+0000 (oldest deadline 2024-05-03T19:03:56.662340+0000) 2024-05-03T19:04:20.802 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:20 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:20.367+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:40.163402+0000 front 2024-05-03T19:03:40.163384+0000 (oldest deadline 2024-05-03T19:04:03.663534+0000) 2024-05-03T19:04:20.802 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:20 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:20.367+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.45:6812 osd.4 since back 2024-05-03T19:03:56.565512+0000 front 2024-05-03T19:03:56.565524+0000 (oldest deadline 2024-05-03T19:04:19.465417+0000) 2024-05-03T19:04:20.998 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:20 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:20.653+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:25.050521+0000 front 2024-05-03T19:03:25.050536+0000 (oldest deadline 2024-05-03T19:03:49.150320+0000) 2024-05-03T19:04:20.999 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:20 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:20.653+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:29.150598+0000 front 2024-05-03T19:03:29.150627+0000 (oldest deadline 2024-05-03T19:03:55.050741+0000) 2024-05-03T19:04:20.999 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:20 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:20.653+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:37.951720+0000 front 2024-05-03T19:03:37.951864+0000 (oldest deadline 2024-05-03T19:04:02.651644+0000) 2024-05-03T19:04:20.999 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:20 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:20.653+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-05-03T19:03:46.653109+0000 front 2024-05-03T19:03:46.653179+0000 (oldest deadline 2024-05-03T19:04:12.553001+0000) 2024-05-03T19:04:21.000 INFO:journalctl@ceph.osd.7.smithi045.stdout:May 03 19:04:20 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-7[51710]: 2024-05-03T19:04:20.670+0000 7fe2c0194700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:32.077137+0000 front 2024-05-03T19:03:32.077303+0000 (oldest deadline 2024-05-03T19:03:53.777433+0000) 2024-05-03T19:04:21.000 INFO:journalctl@ceph.osd.7.smithi045.stdout:May 03 19:04:20 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-7[51710]: 2024-05-03T19:04:20.670+0000 7fe2c0194700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:39.579099+0000 front 2024-05-03T19:03:39.579074+0000 (oldest deadline 2024-05-03T19:04:05.479057+0000) 2024-05-03T19:04:21.000 INFO:journalctl@ceph.osd.7.smithi045.stdout:May 03 19:04:20 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-7[51710]: 2024-05-03T19:04:20.670+0000 7fe2c0194700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-05-03T19:03:45.479498+0000 front 2024-05-03T19:03:45.479594+0000 (oldest deadline 2024-05-03T19:04:10.779647+0000) 2024-05-03T19:04:21.000 INFO:journalctl@ceph.osd.7.smithi045.stdout:May 03 19:04:20 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-7[51710]: 2024-05-03T19:04:20.670+0000 7fe2c0194700 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.45:6812 osd.4 since back 2024-05-03T19:03:54.280511+0000 front 2024-05-03T19:03:54.280483+0000 (oldest deadline 2024-05-03T19:04:18.980674+0000) 2024-05-03T19:04:21.301 INFO:journalctl@ceph.osd.8.smithi113.stdout:May 03 19:04:20 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-8[52667]: 2024-05-03T19:04:20.875+0000 7fb4a8e32700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:21.503361+0000 front 2024-05-03T19:03:21.503321+0000 (oldest deadline 2024-05-03T19:03:49.704026+0000) 2024-05-03T19:04:21.302 INFO:journalctl@ceph.osd.8.smithi113.stdout:May 03 19:04:20 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-8[52667]: 2024-05-03T19:04:20.875+0000 7fb4a8e32700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:30.805173+0000 front 2024-05-03T19:03:30.805157+0000 (oldest deadline 2024-05-03T19:03:53.605897+0000) 2024-05-03T19:04:21.302 INFO:journalctl@ceph.osd.8.smithi113.stdout:May 03 19:04:20 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-8[52667]: 2024-05-03T19:04:20.875+0000 7fb4a8e32700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:37.106650+0000 front 2024-05-03T19:03:37.106721+0000 (oldest deadline 2024-05-03T19:04:02.406804+0000) 2024-05-03T19:04:21.302 INFO:journalctl@ceph.osd.8.smithi113.stdout:May 03 19:04:20 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-8[52667]: 2024-05-03T19:04:20.875+0000 7fb4a8e32700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.45:6812 osd.4 since back 2024-05-03T19:03:57.009866+0000 front 2024-05-03T19:03:57.009783+0000 (oldest deadline 2024-05-03T19:04:18.709619+0000) 2024-05-03T19:04:21.568 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:21 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:04:21.068+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:23.927793+0000 front 2024-05-03T19:03:23.927723+0000 (oldest deadline 2024-05-03T19:03:46.227921+0000) 2024-05-03T19:04:21.569 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:21 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:04:21.068+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:29.729118+0000 front 2024-05-03T19:03:29.729178+0000 (oldest deadline 2024-05-03T19:03:54.429041+0000) 2024-05-03T19:04:21.569 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:21 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:04:21.068+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:40.231110+0000 front 2024-05-03T19:03:40.230916+0000 (oldest deadline 2024-05-03T19:04:03.130962+0000) 2024-05-03T19:04:21.569 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:21 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:04:21.068+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-05-03T19:03:46.032125+0000 front 2024-05-03T19:03:46.032053+0000 (oldest deadline 2024-05-03T19:04:11.332075+0000) 2024-05-03T19:04:21.801 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:21 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:21.413+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:25.060144+0000 front 2024-05-03T19:03:25.060125+0000 (oldest deadline 2024-05-03T19:03:46.760321+0000) 2024-05-03T19:04:21.802 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:21 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:21.413+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:31.362284+0000 front 2024-05-03T19:03:31.362156+0000 (oldest deadline 2024-05-03T19:03:56.662340+0000) 2024-05-03T19:04:21.802 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:21 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:21.413+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:40.163402+0000 front 2024-05-03T19:03:40.163384+0000 (oldest deadline 2024-05-03T19:04:03.663534+0000) 2024-05-03T19:04:21.802 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:21 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:21.413+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.45:6812 osd.4 since back 2024-05-03T19:03:56.565512+0000 front 2024-05-03T19:03:56.565524+0000 (oldest deadline 2024-05-03T19:04:19.465417+0000) 2024-05-03T19:04:21.942 INFO:journalctl@ceph.osd.7.smithi045.stdout:May 03 19:04:21 smithi045 podman[68919]: 2024-05-03 19:04:21.659595363 +0000 UTC m=+5.445382745 container died f9a505103f4fcc6f28681c0246fc917564fd2ea512aa3ff5f078a832f622a22b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-7, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240430, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-05-03T19:04:21.942 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:21 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:21.630+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:25.050521+0000 front 2024-05-03T19:03:25.050536+0000 (oldest deadline 2024-05-03T19:03:49.150320+0000) 2024-05-03T19:04:21.942 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:21 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:21.630+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:29.150598+0000 front 2024-05-03T19:03:29.150627+0000 (oldest deadline 2024-05-03T19:03:55.050741+0000) 2024-05-03T19:04:21.942 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:21 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:21.630+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:37.951720+0000 front 2024-05-03T19:03:37.951864+0000 (oldest deadline 2024-05-03T19:04:02.651644+0000) 2024-05-03T19:04:21.943 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:21 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:21.630+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-05-03T19:03:46.653109+0000 front 2024-05-03T19:03:46.653179+0000 (oldest deadline 2024-05-03T19:04:12.553001+0000) 2024-05-03T19:04:21.943 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:21 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:21.630+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.45:6812 osd.4 since back 2024-05-03T19:03:54.853856+0000 front 2024-05-03T19:03:54.853882+0000 (oldest deadline 2024-05-03T19:04:20.754013+0000) 2024-05-03T19:04:22.248 INFO:journalctl@ceph.osd.7.smithi045.stdout:May 03 19:04:22 smithi045 podman[68919]: 2024-05-03 19:04:22.021030133 +0000 UTC m=+5.806817514 container cleanup f9a505103f4fcc6f28681c0246fc917564fd2ea512aa3ff5f078a832f622a22b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-7, org.label-schema.build-date=20240430, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-05-03T19:04:22.248 INFO:journalctl@ceph.osd.7.smithi045.stdout:May 03 19:04:22 smithi045 bash[68919]: ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-7 2024-05-03T19:04:22.301 INFO:journalctl@ceph.osd.8.smithi113.stdout:May 03 19:04:21 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-8[52667]: 2024-05-03T19:04:21.878+0000 7fb4a8e32700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:21.503361+0000 front 2024-05-03T19:03:21.503321+0000 (oldest deadline 2024-05-03T19:03:49.704026+0000) 2024-05-03T19:04:22.302 INFO:journalctl@ceph.osd.8.smithi113.stdout:May 03 19:04:21 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-8[52667]: 2024-05-03T19:04:21.878+0000 7fb4a8e32700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:30.805173+0000 front 2024-05-03T19:03:30.805157+0000 (oldest deadline 2024-05-03T19:03:53.605897+0000) 2024-05-03T19:04:22.302 INFO:journalctl@ceph.osd.8.smithi113.stdout:May 03 19:04:21 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-8[52667]: 2024-05-03T19:04:21.878+0000 7fb4a8e32700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:37.106650+0000 front 2024-05-03T19:03:37.106721+0000 (oldest deadline 2024-05-03T19:04:02.406804+0000) 2024-05-03T19:04:22.302 INFO:journalctl@ceph.osd.8.smithi113.stdout:May 03 19:04:21 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-8[52667]: 2024-05-03T19:04:21.878+0000 7fb4a8e32700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.45:6812 osd.4 since back 2024-05-03T19:03:57.009866+0000 front 2024-05-03T19:03:57.009783+0000 (oldest deadline 2024-05-03T19:04:18.709619+0000) 2024-05-03T19:04:22.318 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:22 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:04:22.023+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:23.927793+0000 front 2024-05-03T19:03:23.927723+0000 (oldest deadline 2024-05-03T19:03:46.227921+0000) 2024-05-03T19:04:22.319 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:22 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:04:22.023+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:29.729118+0000 front 2024-05-03T19:03:29.729178+0000 (oldest deadline 2024-05-03T19:03:54.429041+0000) 2024-05-03T19:04:22.319 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:22 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:04:22.023+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:40.231110+0000 front 2024-05-03T19:03:40.230916+0000 (oldest deadline 2024-05-03T19:04:03.130962+0000) 2024-05-03T19:04:22.319 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:22 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:04:22.023+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-05-03T19:03:46.032125+0000 front 2024-05-03T19:03:46.032053+0000 (oldest deadline 2024-05-03T19:04:11.332075+0000) 2024-05-03T19:04:22.590 INFO:journalctl@ceph.osd.7.smithi045.stdout:May 03 19:04:22 smithi045 podman[68936]: 2024-05-03 19:04:22.322139643 +0000 UTC m=+0.654824457 container remove f9a505103f4fcc6f28681c0246fc917564fd2ea512aa3ff5f078a832f622a22b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-7, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.build-date=20240430, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-03T19:04:22.801 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:22 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:22.395+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:25.060144+0000 front 2024-05-03T19:03:25.060125+0000 (oldest deadline 2024-05-03T19:03:46.760321+0000) 2024-05-03T19:04:22.802 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:22 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:22.395+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:31.362284+0000 front 2024-05-03T19:03:31.362156+0000 (oldest deadline 2024-05-03T19:03:56.662340+0000) 2024-05-03T19:04:22.802 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:22 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:22.395+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:40.163402+0000 front 2024-05-03T19:03:40.163384+0000 (oldest deadline 2024-05-03T19:04:03.663534+0000) 2024-05-03T19:04:22.802 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:22 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:22.395+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.45:6812 osd.4 since back 2024-05-03T19:03:56.565512+0000 front 2024-05-03T19:03:56.565524+0000 (oldest deadline 2024-05-03T19:04:19.465417+0000) 2024-05-03T19:04:22.876 INFO:journalctl@ceph.osd.7.smithi045.stdout:May 03 19:04:22 smithi045 podman[69025]: 2024-05-03 19:04:22.554932081 +0000 UTC m=+0.018620802 image pull ee4d846202496ba6096593e8e8becc0700a5eb7ebd95536031187f5f337d4012 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd 2024-05-03T19:04:22.877 INFO:journalctl@ceph.osd.7.smithi045.stdout:May 03 19:04:22 smithi045 podman[69025]: 2024-05-03 19:04:22.664846544 +0000 UTC m=+0.128535245 container create b993a57d294545b853986d29cb47f37cef17e7020cc365c6e267c7ceaf4fd48a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-7-deactivate, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240430, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-05-03T19:04:22.877 INFO:journalctl@ceph.osd.7.smithi045.stdout:May 03 19:04:22 smithi045 podman[69025]: 2024-05-03 19:04:22.875667564 +0000 UTC m=+0.339356274 container init b993a57d294545b853986d29cb47f37cef17e7020cc365c6e267c7ceaf4fd48a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-7-deactivate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240430, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-05-03T19:04:22.877 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:22 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:22.588+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:25.050521+0000 front 2024-05-03T19:03:25.050536+0000 (oldest deadline 2024-05-03T19:03:49.150320+0000) 2024-05-03T19:04:22.877 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:22 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:22.588+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:29.150598+0000 front 2024-05-03T19:03:29.150627+0000 (oldest deadline 2024-05-03T19:03:55.050741+0000) 2024-05-03T19:04:22.878 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:22 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:22.588+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:37.951720+0000 front 2024-05-03T19:03:37.951864+0000 (oldest deadline 2024-05-03T19:04:02.651644+0000) 2024-05-03T19:04:22.878 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:22 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:22.588+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-05-03T19:03:46.653109+0000 front 2024-05-03T19:03:46.653179+0000 (oldest deadline 2024-05-03T19:04:12.553001+0000) 2024-05-03T19:04:22.878 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:22 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:22.588+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.45:6812 osd.4 since back 2024-05-03T19:03:54.853856+0000 front 2024-05-03T19:03:54.853882+0000 (oldest deadline 2024-05-03T19:04:20.754013+0000) 2024-05-03T19:04:23.184 INFO:journalctl@ceph.osd.7.smithi045.stdout:May 03 19:04:22 smithi045 podman[69025]: 2024-05-03 19:04:22.878945595 +0000 UTC m=+0.342634307 container start b993a57d294545b853986d29cb47f37cef17e7020cc365c6e267c7ceaf4fd48a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-7-deactivate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240430, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-05-03T19:04:23.184 INFO:journalctl@ceph.osd.7.smithi045.stdout:May 03 19:04:22 smithi045 podman[69025]: 2024-05-03 19:04:22.958939122 +0000 UTC m=+0.422627835 container attach b993a57d294545b853986d29cb47f37cef17e7020cc365c6e267c7ceaf4fd48a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-7-deactivate, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240430, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, ceph=True) 2024-05-03T19:04:23.301 INFO:journalctl@ceph.osd.8.smithi113.stdout:May 03 19:04:22 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-8[52667]: 2024-05-03T19:04:22.900+0000 7fb4a8e32700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:21.503361+0000 front 2024-05-03T19:03:21.503321+0000 (oldest deadline 2024-05-03T19:03:49.704026+0000) 2024-05-03T19:04:23.302 INFO:journalctl@ceph.osd.8.smithi113.stdout:May 03 19:04:22 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-8[52667]: 2024-05-03T19:04:22.900+0000 7fb4a8e32700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:30.805173+0000 front 2024-05-03T19:03:30.805157+0000 (oldest deadline 2024-05-03T19:03:53.605897+0000) 2024-05-03T19:04:23.302 INFO:journalctl@ceph.osd.8.smithi113.stdout:May 03 19:04:22 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-8[52667]: 2024-05-03T19:04:22.900+0000 7fb4a8e32700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:37.106650+0000 front 2024-05-03T19:03:37.106721+0000 (oldest deadline 2024-05-03T19:04:02.406804+0000) 2024-05-03T19:04:23.302 INFO:journalctl@ceph.osd.8.smithi113.stdout:May 03 19:04:22 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-8[52667]: 2024-05-03T19:04:22.900+0000 7fb4a8e32700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.45:6812 osd.4 since back 2024-05-03T19:03:57.009866+0000 front 2024-05-03T19:03:57.009783+0000 (oldest deadline 2024-05-03T19:04:18.709619+0000) 2024-05-03T19:04:23.318 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:22 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:04:22.988+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:23.927793+0000 front 2024-05-03T19:03:23.927723+0000 (oldest deadline 2024-05-03T19:03:46.227921+0000) 2024-05-03T19:04:23.319 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:22 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:04:22.988+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:29.729118+0000 front 2024-05-03T19:03:29.729178+0000 (oldest deadline 2024-05-03T19:03:54.429041+0000) 2024-05-03T19:04:23.319 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:22 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:04:22.988+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:40.231110+0000 front 2024-05-03T19:03:40.230916+0000 (oldest deadline 2024-05-03T19:04:03.130962+0000) 2024-05-03T19:04:23.319 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:22 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:04:22.988+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-05-03T19:03:46.032125+0000 front 2024-05-03T19:03:46.032053+0000 (oldest deadline 2024-05-03T19:04:11.332075+0000) 2024-05-03T19:04:23.468 INFO:journalctl@ceph.osd.7.smithi045.stdout:May 03 19:04:23 smithi045 podman[69025]: 2024-05-03 19:04:23.183897891 +0000 UTC m=+0.647586592 container died b993a57d294545b853986d29cb47f37cef17e7020cc365c6e267c7ceaf4fd48a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-7-deactivate, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240430) 2024-05-03T19:04:23.748 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:23 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:23.559+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:25.050521+0000 front 2024-05-03T19:03:25.050536+0000 (oldest deadline 2024-05-03T19:03:49.150320+0000) 2024-05-03T19:04:23.748 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:23 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:23.559+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:29.150598+0000 front 2024-05-03T19:03:29.150627+0000 (oldest deadline 2024-05-03T19:03:55.050741+0000) 2024-05-03T19:04:23.748 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:23 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:23.559+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:37.951720+0000 front 2024-05-03T19:03:37.951864+0000 (oldest deadline 2024-05-03T19:04:02.651644+0000) 2024-05-03T19:04:23.748 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:23 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:23.559+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-05-03T19:03:46.653109+0000 front 2024-05-03T19:03:46.653179+0000 (oldest deadline 2024-05-03T19:04:12.553001+0000) 2024-05-03T19:04:23.749 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:23 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:23.559+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.45:6812 osd.4 since back 2024-05-03T19:03:54.853856+0000 front 2024-05-03T19:03:54.853882+0000 (oldest deadline 2024-05-03T19:04:20.754013+0000) 2024-05-03T19:04:23.801 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:23 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:23.383+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:25.060144+0000 front 2024-05-03T19:03:25.060125+0000 (oldest deadline 2024-05-03T19:03:46.760321+0000) 2024-05-03T19:04:23.802 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:23 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:23.383+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:31.362284+0000 front 2024-05-03T19:03:31.362156+0000 (oldest deadline 2024-05-03T19:03:56.662340+0000) 2024-05-03T19:04:23.802 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:23 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:23.383+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:40.163402+0000 front 2024-05-03T19:03:40.163384+0000 (oldest deadline 2024-05-03T19:04:03.663534+0000) 2024-05-03T19:04:23.802 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:23 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:23.383+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.45:6812 osd.4 since back 2024-05-03T19:03:56.565512+0000 front 2024-05-03T19:03:56.565524+0000 (oldest deadline 2024-05-03T19:04:19.465417+0000) 2024-05-03T19:04:23.852 DEBUG:teuthology.orchestra.run.smithi045:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f500455e-097b-11ef-bc97-c7b262605968@osd.7.service' 2024-05-03T19:04:23.891 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-03T19:04:23.892 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-05-03T19:04:23.892 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-05-03T19:04:23.892 DEBUG:teuthology.orchestra.run.smithi113:> sudo systemctl stop ceph-f500455e-097b-11ef-bc97-c7b262605968@osd.8 2024-05-03T19:04:24.301 INFO:journalctl@ceph.osd.8.smithi113.stdout:May 03 19:04:23 smithi113 systemd[1]: Stopping Ceph osd.8 for f500455e-097b-11ef-bc97-c7b262605968... 2024-05-03T19:04:24.302 INFO:journalctl@ceph.osd.8.smithi113.stdout:May 03 19:04:23 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-8[52667]: 2024-05-03T19:04:23.944+0000 7fb4a8e32700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:21.503361+0000 front 2024-05-03T19:03:21.503321+0000 (oldest deadline 2024-05-03T19:03:49.704026+0000) 2024-05-03T19:04:24.302 INFO:journalctl@ceph.osd.8.smithi113.stdout:May 03 19:04:23 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-8[52667]: 2024-05-03T19:04:23.944+0000 7fb4a8e32700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:30.805173+0000 front 2024-05-03T19:03:30.805157+0000 (oldest deadline 2024-05-03T19:03:53.605897+0000) 2024-05-03T19:04:24.302 INFO:journalctl@ceph.osd.8.smithi113.stdout:May 03 19:04:23 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-8[52667]: 2024-05-03T19:04:23.944+0000 7fb4a8e32700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:37.106650+0000 front 2024-05-03T19:03:37.106721+0000 (oldest deadline 2024-05-03T19:04:02.406804+0000) 2024-05-03T19:04:24.302 INFO:journalctl@ceph.osd.8.smithi113.stdout:May 03 19:04:23 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-8[52667]: 2024-05-03T19:04:23.944+0000 7fb4a8e32700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.45:6812 osd.4 since back 2024-05-03T19:03:57.009866+0000 front 2024-05-03T19:03:57.009783+0000 (oldest deadline 2024-05-03T19:04:18.709619+0000) 2024-05-03T19:04:24.318 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:24 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:04:24.026+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:23.927793+0000 front 2024-05-03T19:03:23.927723+0000 (oldest deadline 2024-05-03T19:03:46.227921+0000) 2024-05-03T19:04:24.318 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:24 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:04:24.026+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:29.729118+0000 front 2024-05-03T19:03:29.729178+0000 (oldest deadline 2024-05-03T19:03:54.429041+0000) 2024-05-03T19:04:24.319 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:24 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:04:24.026+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:40.231110+0000 front 2024-05-03T19:03:40.230916+0000 (oldest deadline 2024-05-03T19:04:03.130962+0000) 2024-05-03T19:04:24.319 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:24 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:04:24.026+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-05-03T19:03:46.032125+0000 front 2024-05-03T19:03:46.032053+0000 (oldest deadline 2024-05-03T19:04:11.332075+0000) 2024-05-03T19:04:24.748 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:24 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:24.594+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:25.050521+0000 front 2024-05-03T19:03:25.050536+0000 (oldest deadline 2024-05-03T19:03:49.150320+0000) 2024-05-03T19:04:24.748 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:24 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:24.594+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:29.150598+0000 front 2024-05-03T19:03:29.150627+0000 (oldest deadline 2024-05-03T19:03:55.050741+0000) 2024-05-03T19:04:24.749 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:24 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:24.594+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:37.951720+0000 front 2024-05-03T19:03:37.951864+0000 (oldest deadline 2024-05-03T19:04:02.651644+0000) 2024-05-03T19:04:24.749 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:24 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:24.594+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-05-03T19:03:46.653109+0000 front 2024-05-03T19:03:46.653179+0000 (oldest deadline 2024-05-03T19:04:12.553001+0000) 2024-05-03T19:04:24.749 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:24 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:24.594+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.45:6812 osd.4 since back 2024-05-03T19:03:54.853856+0000 front 2024-05-03T19:03:54.853882+0000 (oldest deadline 2024-05-03T19:04:20.754013+0000) 2024-05-03T19:04:24.801 INFO:journalctl@ceph.osd.8.smithi113.stdout:May 03 19:04:24 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-8[52667]: 2024-05-03T19:04:24.507+0000 7fb4ac83e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-03T19:04:24.802 INFO:journalctl@ceph.osd.8.smithi113.stdout:May 03 19:04:24 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-8[52667]: 2024-05-03T19:04:24.507+0000 7fb4ac83e700 -1 osd.8 85 *** Got signal Terminated *** 2024-05-03T19:04:24.802 INFO:journalctl@ceph.osd.8.smithi113.stdout:May 03 19:04:24 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-8[52667]: 2024-05-03T19:04:24.507+0000 7fb4ac83e700 -1 osd.8 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-03T19:04:24.802 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:24 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:24.372+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:25.060144+0000 front 2024-05-03T19:03:25.060125+0000 (oldest deadline 2024-05-03T19:03:46.760321+0000) 2024-05-03T19:04:24.802 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:24 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:24.372+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:31.362284+0000 front 2024-05-03T19:03:31.362156+0000 (oldest deadline 2024-05-03T19:03:56.662340+0000) 2024-05-03T19:04:24.802 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:24 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:24.372+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:40.163402+0000 front 2024-05-03T19:03:40.163384+0000 (oldest deadline 2024-05-03T19:04:03.663534+0000) 2024-05-03T19:04:24.803 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:24 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:24.372+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.45:6812 osd.4 since back 2024-05-03T19:03:56.565512+0000 front 2024-05-03T19:03:56.565524+0000 (oldest deadline 2024-05-03T19:04:19.465417+0000) 2024-05-03T19:04:25.301 INFO:journalctl@ceph.osd.8.smithi113.stdout:May 03 19:04:24 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-8[52667]: 2024-05-03T19:04:24.985+0000 7fb4a8e32700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:21.503361+0000 front 2024-05-03T19:03:21.503321+0000 (oldest deadline 2024-05-03T19:03:49.704026+0000) 2024-05-03T19:04:25.301 INFO:journalctl@ceph.osd.8.smithi113.stdout:May 03 19:04:24 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-8[52667]: 2024-05-03T19:04:24.985+0000 7fb4a8e32700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:30.805173+0000 front 2024-05-03T19:03:30.805157+0000 (oldest deadline 2024-05-03T19:03:53.605897+0000) 2024-05-03T19:04:25.302 INFO:journalctl@ceph.osd.8.smithi113.stdout:May 03 19:04:24 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-8[52667]: 2024-05-03T19:04:24.985+0000 7fb4a8e32700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:37.106650+0000 front 2024-05-03T19:03:37.106721+0000 (oldest deadline 2024-05-03T19:04:02.406804+0000) 2024-05-03T19:04:25.302 INFO:journalctl@ceph.osd.8.smithi113.stdout:May 03 19:04:24 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-8[52667]: 2024-05-03T19:04:24.985+0000 7fb4a8e32700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.45:6812 osd.4 since back 2024-05-03T19:03:57.009866+0000 front 2024-05-03T19:03:57.009783+0000 (oldest deadline 2024-05-03T19:04:18.709619+0000) 2024-05-03T19:04:25.568 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:25 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:04:25.072+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:23.927793+0000 front 2024-05-03T19:03:23.927723+0000 (oldest deadline 2024-05-03T19:03:46.227921+0000) 2024-05-03T19:04:25.569 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:25 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:04:25.072+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:29.729118+0000 front 2024-05-03T19:03:29.729178+0000 (oldest deadline 2024-05-03T19:03:54.429041+0000) 2024-05-03T19:04:25.569 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:25 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:04:25.072+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:40.231110+0000 front 2024-05-03T19:03:40.230916+0000 (oldest deadline 2024-05-03T19:04:03.130962+0000) 2024-05-03T19:04:25.570 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:25 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:04:25.072+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-05-03T19:03:46.032125+0000 front 2024-05-03T19:03:46.032053+0000 (oldest deadline 2024-05-03T19:04:11.332075+0000) 2024-05-03T19:04:25.802 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:25 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:25.402+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:25.060144+0000 front 2024-05-03T19:03:25.060125+0000 (oldest deadline 2024-05-03T19:03:46.760321+0000) 2024-05-03T19:04:25.802 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:25 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:25.402+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:31.362284+0000 front 2024-05-03T19:03:31.362156+0000 (oldest deadline 2024-05-03T19:03:56.662340+0000) 2024-05-03T19:04:25.802 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:25 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:25.402+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:40.163402+0000 front 2024-05-03T19:03:40.163384+0000 (oldest deadline 2024-05-03T19:04:03.663534+0000) 2024-05-03T19:04:25.803 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:25 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:25.402+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.45:6812 osd.4 since back 2024-05-03T19:03:56.565512+0000 front 2024-05-03T19:03:56.565524+0000 (oldest deadline 2024-05-03T19:04:19.465417+0000) 2024-05-03T19:04:25.998 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:25 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:25.619+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:25.050521+0000 front 2024-05-03T19:03:25.050536+0000 (oldest deadline 2024-05-03T19:03:49.150320+0000) 2024-05-03T19:04:25.998 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:25 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:25.619+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:29.150598+0000 front 2024-05-03T19:03:29.150627+0000 (oldest deadline 2024-05-03T19:03:55.050741+0000) 2024-05-03T19:04:25.999 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:25 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:25.619+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:37.951720+0000 front 2024-05-03T19:03:37.951864+0000 (oldest deadline 2024-05-03T19:04:02.651644+0000) 2024-05-03T19:04:25.999 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:25 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:25.619+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-05-03T19:03:46.653109+0000 front 2024-05-03T19:03:46.653179+0000 (oldest deadline 2024-05-03T19:04:12.553001+0000) 2024-05-03T19:04:25.999 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:25 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:25.619+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.45:6812 osd.4 since back 2024-05-03T19:03:54.853856+0000 front 2024-05-03T19:03:54.853882+0000 (oldest deadline 2024-05-03T19:04:20.754013+0000) 2024-05-03T19:04:26.301 INFO:journalctl@ceph.osd.8.smithi113.stdout:May 03 19:04:25 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-8[52667]: 2024-05-03T19:04:25.986+0000 7fb4a8e32700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:21.503361+0000 front 2024-05-03T19:03:21.503321+0000 (oldest deadline 2024-05-03T19:03:49.704026+0000) 2024-05-03T19:04:26.301 INFO:journalctl@ceph.osd.8.smithi113.stdout:May 03 19:04:25 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-8[52667]: 2024-05-03T19:04:25.986+0000 7fb4a8e32700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:30.805173+0000 front 2024-05-03T19:03:30.805157+0000 (oldest deadline 2024-05-03T19:03:53.605897+0000) 2024-05-03T19:04:26.302 INFO:journalctl@ceph.osd.8.smithi113.stdout:May 03 19:04:25 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-8[52667]: 2024-05-03T19:04:25.986+0000 7fb4a8e32700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:37.106650+0000 front 2024-05-03T19:03:37.106721+0000 (oldest deadline 2024-05-03T19:04:02.406804+0000) 2024-05-03T19:04:26.302 INFO:journalctl@ceph.osd.8.smithi113.stdout:May 03 19:04:25 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-8[52667]: 2024-05-03T19:04:25.986+0000 7fb4a8e32700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.45:6812 osd.4 since back 2024-05-03T19:03:57.009866+0000 front 2024-05-03T19:03:57.009783+0000 (oldest deadline 2024-05-03T19:04:18.709619+0000) 2024-05-03T19:04:26.318 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:26 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:04:26.076+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:23.927793+0000 front 2024-05-03T19:03:23.927723+0000 (oldest deadline 2024-05-03T19:03:46.227921+0000) 2024-05-03T19:04:26.319 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:26 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:04:26.076+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:29.729118+0000 front 2024-05-03T19:03:29.729178+0000 (oldest deadline 2024-05-03T19:03:54.429041+0000) 2024-05-03T19:04:26.319 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:26 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:04:26.076+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:40.231110+0000 front 2024-05-03T19:03:40.230916+0000 (oldest deadline 2024-05-03T19:04:03.130962+0000) 2024-05-03T19:04:26.319 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:26 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:04:26.076+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-05-03T19:03:46.032125+0000 front 2024-05-03T19:03:46.032053+0000 (oldest deadline 2024-05-03T19:04:11.332075+0000) 2024-05-03T19:04:26.801 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:26 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:26.409+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:25.060144+0000 front 2024-05-03T19:03:25.060125+0000 (oldest deadline 2024-05-03T19:03:46.760321+0000) 2024-05-03T19:04:26.801 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:26 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:26.409+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:31.362284+0000 front 2024-05-03T19:03:31.362156+0000 (oldest deadline 2024-05-03T19:03:56.662340+0000) 2024-05-03T19:04:26.802 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:26 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:26.409+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:40.163402+0000 front 2024-05-03T19:03:40.163384+0000 (oldest deadline 2024-05-03T19:04:03.663534+0000) 2024-05-03T19:04:26.802 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:26 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:26.409+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.45:6812 osd.4 since back 2024-05-03T19:03:56.565512+0000 front 2024-05-03T19:03:56.565524+0000 (oldest deadline 2024-05-03T19:04:19.465417+0000) 2024-05-03T19:04:26.998 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:26 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:26.601+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:25.050521+0000 front 2024-05-03T19:03:25.050536+0000 (oldest deadline 2024-05-03T19:03:49.150320+0000) 2024-05-03T19:04:26.998 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:26 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:26.601+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:29.150598+0000 front 2024-05-03T19:03:29.150627+0000 (oldest deadline 2024-05-03T19:03:55.050741+0000) 2024-05-03T19:04:26.998 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:26 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:26.601+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:37.951720+0000 front 2024-05-03T19:03:37.951864+0000 (oldest deadline 2024-05-03T19:04:02.651644+0000) 2024-05-03T19:04:26.999 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:26 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:26.601+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-05-03T19:03:46.653109+0000 front 2024-05-03T19:03:46.653179+0000 (oldest deadline 2024-05-03T19:04:12.553001+0000) 2024-05-03T19:04:26.999 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:26 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:26.601+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.45:6812 osd.4 since back 2024-05-03T19:03:54.853856+0000 front 2024-05-03T19:03:54.853882+0000 (oldest deadline 2024-05-03T19:04:20.754013+0000) 2024-05-03T19:04:27.301 INFO:journalctl@ceph.osd.8.smithi113.stdout:May 03 19:04:27 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-8[52667]: 2024-05-03T19:04:27.023+0000 7fb4a8e32700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:21.503361+0000 front 2024-05-03T19:03:21.503321+0000 (oldest deadline 2024-05-03T19:03:49.704026+0000) 2024-05-03T19:04:27.301 INFO:journalctl@ceph.osd.8.smithi113.stdout:May 03 19:04:27 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-8[52667]: 2024-05-03T19:04:27.023+0000 7fb4a8e32700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:30.805173+0000 front 2024-05-03T19:03:30.805157+0000 (oldest deadline 2024-05-03T19:03:53.605897+0000) 2024-05-03T19:04:27.302 INFO:journalctl@ceph.osd.8.smithi113.stdout:May 03 19:04:27 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-8[52667]: 2024-05-03T19:04:27.023+0000 7fb4a8e32700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:37.106650+0000 front 2024-05-03T19:03:37.106721+0000 (oldest deadline 2024-05-03T19:04:02.406804+0000) 2024-05-03T19:04:27.302 INFO:journalctl@ceph.osd.8.smithi113.stdout:May 03 19:04:27 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-8[52667]: 2024-05-03T19:04:27.023+0000 7fb4a8e32700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.45:6812 osd.4 since back 2024-05-03T19:03:57.009866+0000 front 2024-05-03T19:03:57.009783+0000 (oldest deadline 2024-05-03T19:04:18.709619+0000) 2024-05-03T19:04:27.568 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:27 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:04:27.093+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:23.927793+0000 front 2024-05-03T19:03:23.927723+0000 (oldest deadline 2024-05-03T19:03:46.227921+0000) 2024-05-03T19:04:27.569 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:27 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:04:27.093+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:29.729118+0000 front 2024-05-03T19:03:29.729178+0000 (oldest deadline 2024-05-03T19:03:54.429041+0000) 2024-05-03T19:04:27.569 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:27 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:04:27.093+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:40.231110+0000 front 2024-05-03T19:03:40.230916+0000 (oldest deadline 2024-05-03T19:04:03.130962+0000) 2024-05-03T19:04:27.569 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:27 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:04:27.093+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-05-03T19:03:46.032125+0000 front 2024-05-03T19:03:46.032053+0000 (oldest deadline 2024-05-03T19:04:11.332075+0000) 2024-05-03T19:04:27.801 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:27 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:27.400+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:25.060144+0000 front 2024-05-03T19:03:25.060125+0000 (oldest deadline 2024-05-03T19:03:46.760321+0000) 2024-05-03T19:04:27.802 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:27 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:27.400+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:31.362284+0000 front 2024-05-03T19:03:31.362156+0000 (oldest deadline 2024-05-03T19:03:56.662340+0000) 2024-05-03T19:04:27.802 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:27 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:27.400+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:40.163402+0000 front 2024-05-03T19:03:40.163384+0000 (oldest deadline 2024-05-03T19:04:03.663534+0000) 2024-05-03T19:04:27.802 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:27 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:27.400+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.45:6812 osd.4 since back 2024-05-03T19:03:56.565512+0000 front 2024-05-03T19:03:56.565524+0000 (oldest deadline 2024-05-03T19:04:19.465417+0000) 2024-05-03T19:04:27.900 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:27 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:27.562+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:25.050521+0000 front 2024-05-03T19:03:25.050536+0000 (oldest deadline 2024-05-03T19:03:49.150320+0000) 2024-05-03T19:04:27.900 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:27 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:27.562+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:29.150598+0000 front 2024-05-03T19:03:29.150627+0000 (oldest deadline 2024-05-03T19:03:55.050741+0000) 2024-05-03T19:04:27.900 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:27 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:27.562+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:37.951720+0000 front 2024-05-03T19:03:37.951864+0000 (oldest deadline 2024-05-03T19:04:02.651644+0000) 2024-05-03T19:04:27.900 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:27 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:27.562+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-05-03T19:03:46.653109+0000 front 2024-05-03T19:03:46.653179+0000 (oldest deadline 2024-05-03T19:04:12.553001+0000) 2024-05-03T19:04:27.900 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:27 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:27.562+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.45:6812 osd.4 since back 2024-05-03T19:03:54.853856+0000 front 2024-05-03T19:03:54.853882+0000 (oldest deadline 2024-05-03T19:04:20.754013+0000) 2024-05-03T19:04:27.901 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:27 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:27.562+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.113:6812 osd.5 since back 2024-05-03T19:04:04.755716+0000 front 2024-05-03T19:04:04.755535+0000 (oldest deadline 2024-05-03T19:04:27.555916+0000) 2024-05-03T19:04:28.301 INFO:journalctl@ceph.osd.8.smithi113.stdout:May 03 19:04:28 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-8[52667]: 2024-05-03T19:04:28.048+0000 7fb4a8e32700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:21.503361+0000 front 2024-05-03T19:03:21.503321+0000 (oldest deadline 2024-05-03T19:03:49.704026+0000) 2024-05-03T19:04:28.302 INFO:journalctl@ceph.osd.8.smithi113.stdout:May 03 19:04:28 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-8[52667]: 2024-05-03T19:04:28.048+0000 7fb4a8e32700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:30.805173+0000 front 2024-05-03T19:03:30.805157+0000 (oldest deadline 2024-05-03T19:03:53.605897+0000) 2024-05-03T19:04:28.302 INFO:journalctl@ceph.osd.8.smithi113.stdout:May 03 19:04:28 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-8[52667]: 2024-05-03T19:04:28.048+0000 7fb4a8e32700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:37.106650+0000 front 2024-05-03T19:03:37.106721+0000 (oldest deadline 2024-05-03T19:04:02.406804+0000) 2024-05-03T19:04:28.302 INFO:journalctl@ceph.osd.8.smithi113.stdout:May 03 19:04:28 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-8[52667]: 2024-05-03T19:04:28.048+0000 7fb4a8e32700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.45:6812 osd.4 since back 2024-05-03T19:03:57.009866+0000 front 2024-05-03T19:03:57.009783+0000 (oldest deadline 2024-05-03T19:04:18.709619+0000) 2024-05-03T19:04:28.568 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:28 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:04:28.067+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:23.927793+0000 front 2024-05-03T19:03:23.927723+0000 (oldest deadline 2024-05-03T19:03:46.227921+0000) 2024-05-03T19:04:28.569 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:28 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:04:28.067+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:29.729118+0000 front 2024-05-03T19:03:29.729178+0000 (oldest deadline 2024-05-03T19:03:54.429041+0000) 2024-05-03T19:04:28.569 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:28 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:04:28.067+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:40.231110+0000 front 2024-05-03T19:03:40.230916+0000 (oldest deadline 2024-05-03T19:04:03.130962+0000) 2024-05-03T19:04:28.569 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:28 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:04:28.067+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-05-03T19:03:46.032125+0000 front 2024-05-03T19:03:46.032053+0000 (oldest deadline 2024-05-03T19:04:11.332075+0000) 2024-05-03T19:04:28.574 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:28 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:28.379+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:25.060144+0000 front 2024-05-03T19:03:25.060125+0000 (oldest deadline 2024-05-03T19:03:46.760321+0000) 2024-05-03T19:04:28.574 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:28 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:28.379+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:31.362284+0000 front 2024-05-03T19:03:31.362156+0000 (oldest deadline 2024-05-03T19:03:56.662340+0000) 2024-05-03T19:04:28.574 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:28 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:28.379+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:40.163402+0000 front 2024-05-03T19:03:40.163384+0000 (oldest deadline 2024-05-03T19:04:03.663534+0000) 2024-05-03T19:04:28.574 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:28 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:28.379+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.45:6812 osd.4 since back 2024-05-03T19:03:56.565512+0000 front 2024-05-03T19:03:56.565524+0000 (oldest deadline 2024-05-03T19:04:19.465417+0000) 2024-05-03T19:04:28.574 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:28 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:28.379+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.113:6812 osd.5 since back 2024-05-03T19:04:04.567504+0000 front 2024-05-03T19:04:04.567582+0000 (oldest deadline 2024-05-03T19:04:28.067641+0000) 2024-05-03T19:04:28.998 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:28 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:28.517+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:25.050521+0000 front 2024-05-03T19:03:25.050536+0000 (oldest deadline 2024-05-03T19:03:49.150320+0000) 2024-05-03T19:04:28.998 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:28 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:28.517+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:29.150598+0000 front 2024-05-03T19:03:29.150627+0000 (oldest deadline 2024-05-03T19:03:55.050741+0000) 2024-05-03T19:04:28.999 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:28 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:28.517+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:37.951720+0000 front 2024-05-03T19:03:37.951864+0000 (oldest deadline 2024-05-03T19:04:02.651644+0000) 2024-05-03T19:04:28.999 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:28 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:28.517+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-05-03T19:03:46.653109+0000 front 2024-05-03T19:03:46.653179+0000 (oldest deadline 2024-05-03T19:04:12.553001+0000) 2024-05-03T19:04:28.999 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:28 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:28.517+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.45:6812 osd.4 since back 2024-05-03T19:03:54.853856+0000 front 2024-05-03T19:03:54.853882+0000 (oldest deadline 2024-05-03T19:04:20.754013+0000) 2024-05-03T19:04:28.999 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:28 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:28.517+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.113:6812 osd.5 since back 2024-05-03T19:04:04.755716+0000 front 2024-05-03T19:04:04.755535+0000 (oldest deadline 2024-05-03T19:04:27.555916+0000) 2024-05-03T19:04:29.301 INFO:journalctl@ceph.osd.8.smithi113.stdout:May 03 19:04:29 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-8[52667]: 2024-05-03T19:04:29.009+0000 7fb4a8e32700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:21.503361+0000 front 2024-05-03T19:03:21.503321+0000 (oldest deadline 2024-05-03T19:03:49.704026+0000) 2024-05-03T19:04:29.301 INFO:journalctl@ceph.osd.8.smithi113.stdout:May 03 19:04:29 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-8[52667]: 2024-05-03T19:04:29.009+0000 7fb4a8e32700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:30.805173+0000 front 2024-05-03T19:03:30.805157+0000 (oldest deadline 2024-05-03T19:03:53.605897+0000) 2024-05-03T19:04:29.302 INFO:journalctl@ceph.osd.8.smithi113.stdout:May 03 19:04:29 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-8[52667]: 2024-05-03T19:04:29.009+0000 7fb4a8e32700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:37.106650+0000 front 2024-05-03T19:03:37.106721+0000 (oldest deadline 2024-05-03T19:04:02.406804+0000) 2024-05-03T19:04:29.302 INFO:journalctl@ceph.osd.8.smithi113.stdout:May 03 19:04:29 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-8[52667]: 2024-05-03T19:04:29.009+0000 7fb4a8e32700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.45:6812 osd.4 since back 2024-05-03T19:03:57.009866+0000 front 2024-05-03T19:03:57.009783+0000 (oldest deadline 2024-05-03T19:04:18.709619+0000) 2024-05-03T19:04:29.302 INFO:journalctl@ceph.osd.8.smithi113.stdout:May 03 19:04:29 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-8[52667]: 2024-05-03T19:04:29.009+0000 7fb4a8e32700 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.113:6812 osd.5 since back 2024-05-03T19:04:04.610740+0000 front 2024-05-03T19:04:04.610772+0000 (oldest deadline 2024-05-03T19:04:28.710872+0000) 2024-05-03T19:04:29.318 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:29 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:04:29.051+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:23.927793+0000 front 2024-05-03T19:03:23.927723+0000 (oldest deadline 2024-05-03T19:03:46.227921+0000) 2024-05-03T19:04:29.318 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:29 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:04:29.051+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:29.729118+0000 front 2024-05-03T19:03:29.729178+0000 (oldest deadline 2024-05-03T19:03:54.429041+0000) 2024-05-03T19:04:29.319 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:29 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:04:29.051+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:40.231110+0000 front 2024-05-03T19:03:40.230916+0000 (oldest deadline 2024-05-03T19:04:03.130962+0000) 2024-05-03T19:04:29.319 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:29 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:04:29.051+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-05-03T19:03:46.032125+0000 front 2024-05-03T19:03:46.032053+0000 (oldest deadline 2024-05-03T19:04:11.332075+0000) 2024-05-03T19:04:29.319 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:29 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:04:29.051+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.113:6812 osd.5 since back 2024-05-03T19:04:03.034370+0000 front 2024-05-03T19:04:03.034324+0000 (oldest deadline 2024-05-03T19:04:28.334378+0000) 2024-05-03T19:04:29.801 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:29 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:29.375+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:25.060144+0000 front 2024-05-03T19:03:25.060125+0000 (oldest deadline 2024-05-03T19:03:46.760321+0000) 2024-05-03T19:04:29.802 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:29 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:29.375+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:31.362284+0000 front 2024-05-03T19:03:31.362156+0000 (oldest deadline 2024-05-03T19:03:56.662340+0000) 2024-05-03T19:04:29.802 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:29 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:29.375+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:40.163402+0000 front 2024-05-03T19:03:40.163384+0000 (oldest deadline 2024-05-03T19:04:03.663534+0000) 2024-05-03T19:04:29.802 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:29 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:29.375+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.45:6812 osd.4 since back 2024-05-03T19:03:56.565512+0000 front 2024-05-03T19:03:56.565524+0000 (oldest deadline 2024-05-03T19:04:19.465417+0000) 2024-05-03T19:04:29.802 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:29 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:29.375+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.113:6812 osd.5 since back 2024-05-03T19:04:04.567504+0000 front 2024-05-03T19:04:04.567582+0000 (oldest deadline 2024-05-03T19:04:28.067641+0000) 2024-05-03T19:04:29.998 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:29 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:29.523+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:25.050521+0000 front 2024-05-03T19:03:25.050536+0000 (oldest deadline 2024-05-03T19:03:49.150320+0000) 2024-05-03T19:04:29.998 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:29 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:29.523+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:29.150598+0000 front 2024-05-03T19:03:29.150627+0000 (oldest deadline 2024-05-03T19:03:55.050741+0000) 2024-05-03T19:04:29.998 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:29 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:29.523+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:37.951720+0000 front 2024-05-03T19:03:37.951864+0000 (oldest deadline 2024-05-03T19:04:02.651644+0000) 2024-05-03T19:04:29.999 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:29 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:29.523+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-05-03T19:03:46.653109+0000 front 2024-05-03T19:03:46.653179+0000 (oldest deadline 2024-05-03T19:04:12.553001+0000) 2024-05-03T19:04:29.999 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:29 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:29.523+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.45:6812 osd.4 since back 2024-05-03T19:03:54.853856+0000 front 2024-05-03T19:03:54.853882+0000 (oldest deadline 2024-05-03T19:04:20.754013+0000) 2024-05-03T19:04:29.999 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:29 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:29.523+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.113:6812 osd.5 since back 2024-05-03T19:04:04.755716+0000 front 2024-05-03T19:04:04.755535+0000 (oldest deadline 2024-05-03T19:04:27.555916+0000) 2024-05-03T19:04:30.215 INFO:journalctl@ceph.osd.8.smithi113.stdout:May 03 19:04:29 smithi113 podman[70505]: 2024-05-03 19:04:29.893861574 +0000 UTC m=+5.814653279 container died 1e4930a84744cb547050199b3a89a02d2d22fa940241cd93052229b762cc7146 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-8, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240430, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux ) 2024-05-03T19:04:30.215 INFO:journalctl@ceph.osd.8.smithi113.stdout:May 03 19:04:30 smithi113 podman[70505]: 2024-05-03 19:04:30.214559967 +0000 UTC m=+6.135351663 container cleanup 1e4930a84744cb547050199b3a89a02d2d22fa940241cd93052229b762cc7146 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-8, org.label-schema.build-date=20240430, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-05-03T19:04:30.552 INFO:journalctl@ceph.osd.8.smithi113.stdout:May 03 19:04:30 smithi113 bash[70505]: ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-8 2024-05-03T19:04:30.552 INFO:journalctl@ceph.osd.8.smithi113.stdout:May 03 19:04:30 smithi113 podman[70535]: 2024-05-03 19:04:30.432611687 +0000 UTC m=+0.534046065 container remove 1e4930a84744cb547050199b3a89a02d2d22fa940241cd93052229b762cc7146 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-8, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240430) 2024-05-03T19:04:30.552 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:30 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:30.422+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:25.060144+0000 front 2024-05-03T19:03:25.060125+0000 (oldest deadline 2024-05-03T19:03:46.760321+0000) 2024-05-03T19:04:30.552 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:30 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:30.422+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:31.362284+0000 front 2024-05-03T19:03:31.362156+0000 (oldest deadline 2024-05-03T19:03:56.662340+0000) 2024-05-03T19:04:30.552 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:30 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:30.422+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:40.163402+0000 front 2024-05-03T19:03:40.163384+0000 (oldest deadline 2024-05-03T19:04:03.663534+0000) 2024-05-03T19:04:30.552 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:30 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:30.422+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.45:6812 osd.4 since back 2024-05-03T19:03:56.565512+0000 front 2024-05-03T19:03:56.565524+0000 (oldest deadline 2024-05-03T19:04:19.465417+0000) 2024-05-03T19:04:30.553 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:30 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:30.422+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.113:6812 osd.5 since back 2024-05-03T19:04:04.567504+0000 front 2024-05-03T19:04:04.567582+0000 (oldest deadline 2024-05-03T19:04:28.067641+0000) 2024-05-03T19:04:30.568 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:30 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:04:30.076+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:23.927793+0000 front 2024-05-03T19:03:23.927723+0000 (oldest deadline 2024-05-03T19:03:46.227921+0000) 2024-05-03T19:04:30.568 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:30 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:04:30.076+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:29.729118+0000 front 2024-05-03T19:03:29.729178+0000 (oldest deadline 2024-05-03T19:03:54.429041+0000) 2024-05-03T19:04:30.569 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:30 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:04:30.076+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:40.231110+0000 front 2024-05-03T19:03:40.230916+0000 (oldest deadline 2024-05-03T19:04:03.130962+0000) 2024-05-03T19:04:30.569 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:30 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:04:30.076+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-05-03T19:03:46.032125+0000 front 2024-05-03T19:03:46.032053+0000 (oldest deadline 2024-05-03T19:04:11.332075+0000) 2024-05-03T19:04:30.569 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:30 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:04:30.076+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.113:6812 osd.5 since back 2024-05-03T19:04:03.034370+0000 front 2024-05-03T19:04:03.034324+0000 (oldest deadline 2024-05-03T19:04:28.334378+0000) 2024-05-03T19:04:30.998 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:30 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:30.513+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:25.050521+0000 front 2024-05-03T19:03:25.050536+0000 (oldest deadline 2024-05-03T19:03:49.150320+0000) 2024-05-03T19:04:30.998 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:30 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:30.513+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:29.150598+0000 front 2024-05-03T19:03:29.150627+0000 (oldest deadline 2024-05-03T19:03:55.050741+0000) 2024-05-03T19:04:30.998 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:30 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:30.513+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:37.951720+0000 front 2024-05-03T19:03:37.951864+0000 (oldest deadline 2024-05-03T19:04:02.651644+0000) 2024-05-03T19:04:30.998 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:30 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:30.513+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-05-03T19:03:46.653109+0000 front 2024-05-03T19:03:46.653179+0000 (oldest deadline 2024-05-03T19:04:12.553001+0000) 2024-05-03T19:04:30.999 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:30 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:30.513+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.45:6812 osd.4 since back 2024-05-03T19:03:54.853856+0000 front 2024-05-03T19:03:54.853882+0000 (oldest deadline 2024-05-03T19:04:20.754013+0000) 2024-05-03T19:04:30.999 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:30 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:30.513+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.113:6812 osd.5 since back 2024-05-03T19:04:04.755716+0000 front 2024-05-03T19:04:04.755535+0000 (oldest deadline 2024-05-03T19:04:27.555916+0000) 2024-05-03T19:04:31.086 INFO:journalctl@ceph.osd.8.smithi113.stdout:May 03 19:04:30 smithi113 podman[70628]: 2024-05-03 19:04:30.727240627 +0000 UTC m=+0.017997028 image pull ee4d846202496ba6096593e8e8becc0700a5eb7ebd95536031187f5f337d4012 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd 2024-05-03T19:04:31.086 INFO:journalctl@ceph.osd.8.smithi113.stdout:May 03 19:04:30 smithi113 podman[70628]: 2024-05-03 19:04:30.833462317 +0000 UTC m=+0.124218742 container create 31423052372de1d5387daed7e695a9eb172855ac2ce5e7275ee9688a035ae6b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-8-deactivate, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240430, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-03T19:04:31.087 INFO:journalctl@ceph.osd.8.smithi113.stdout:May 03 19:04:31 smithi113 podman[70628]: 2024-05-03 19:04:31.01975499 +0000 UTC m=+0.310511382 container init 31423052372de1d5387daed7e695a9eb172855ac2ce5e7275ee9688a035ae6b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-8-deactivate, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240430, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux ) 2024-05-03T19:04:31.087 INFO:journalctl@ceph.osd.8.smithi113.stdout:May 03 19:04:31 smithi113 podman[70628]: 2024-05-03 19:04:31.023150764 +0000 UTC m=+0.313907149 container start 31423052372de1d5387daed7e695a9eb172855ac2ce5e7275ee9688a035ae6b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-8-deactivate, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240430, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-03T19:04:31.087 INFO:journalctl@ceph.osd.8.smithi113.stdout:May 03 19:04:31 smithi113 podman[70628]: 2024-05-03 19:04:31.085858008 +0000 UTC m=+0.376614407 container attach 31423052372de1d5387daed7e695a9eb172855ac2ce5e7275ee9688a035ae6b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-8-deactivate, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240430, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-05-03T19:04:31.318 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:31 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:04:31.065+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:23.927793+0000 front 2024-05-03T19:03:23.927723+0000 (oldest deadline 2024-05-03T19:03:46.227921+0000) 2024-05-03T19:04:31.319 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:31 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:04:31.065+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:29.729118+0000 front 2024-05-03T19:03:29.729178+0000 (oldest deadline 2024-05-03T19:03:54.429041+0000) 2024-05-03T19:04:31.319 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:31 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:04:31.065+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:40.231110+0000 front 2024-05-03T19:03:40.230916+0000 (oldest deadline 2024-05-03T19:04:03.130962+0000) 2024-05-03T19:04:31.319 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:31 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:04:31.065+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-05-03T19:03:46.032125+0000 front 2024-05-03T19:03:46.032053+0000 (oldest deadline 2024-05-03T19:04:11.332075+0000) 2024-05-03T19:04:31.319 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:31 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:04:31.065+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.113:6812 osd.5 since back 2024-05-03T19:04:03.034370+0000 front 2024-05-03T19:04:03.034324+0000 (oldest deadline 2024-05-03T19:04:28.334378+0000) 2024-05-03T19:04:31.459 INFO:journalctl@ceph.osd.8.smithi113.stdout:May 03 19:04:31 smithi113 podman[70628]: 2024-05-03 19:04:31.320476667 +0000 UTC m=+0.611233056 container died 31423052372de1d5387daed7e695a9eb172855ac2ce5e7275ee9688a035ae6b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-8-deactivate, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240430, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-05-03T19:04:31.802 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:31 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:31.458+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:25.060144+0000 front 2024-05-03T19:03:25.060125+0000 (oldest deadline 2024-05-03T19:03:46.760321+0000) 2024-05-03T19:04:31.802 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:31 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:31.458+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:31.362284+0000 front 2024-05-03T19:03:31.362156+0000 (oldest deadline 2024-05-03T19:03:56.662340+0000) 2024-05-03T19:04:31.802 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:31 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:31.458+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:40.163402+0000 front 2024-05-03T19:03:40.163384+0000 (oldest deadline 2024-05-03T19:04:03.663534+0000) 2024-05-03T19:04:31.802 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:31 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:31.458+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.45:6812 osd.4 since back 2024-05-03T19:03:56.565512+0000 front 2024-05-03T19:03:56.565524+0000 (oldest deadline 2024-05-03T19:04:19.465417+0000) 2024-05-03T19:04:31.802 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:31 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:31.458+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.113:6812 osd.5 since back 2024-05-03T19:04:04.567504+0000 front 2024-05-03T19:04:04.567582+0000 (oldest deadline 2024-05-03T19:04:28.067641+0000) 2024-05-03T19:04:31.875 DEBUG:teuthology.orchestra.run.smithi113:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f500455e-097b-11ef-bc97-c7b262605968@osd.8.service' 2024-05-03T19:04:31.911 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-03T19:04:31.912 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-05-03T19:04:31.912 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-05-03T19:04:31.912 DEBUG:teuthology.orchestra.run.smithi032:> sudo systemctl stop ceph-f500455e-097b-11ef-bc97-c7b262605968@osd.9 2024-05-03T19:04:31.998 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:31 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:31.549+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:25.050521+0000 front 2024-05-03T19:03:25.050536+0000 (oldest deadline 2024-05-03T19:03:49.150320+0000) 2024-05-03T19:04:31.998 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:31 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:31.549+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:29.150598+0000 front 2024-05-03T19:03:29.150627+0000 (oldest deadline 2024-05-03T19:03:55.050741+0000) 2024-05-03T19:04:31.999 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:31 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:31.549+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:37.951720+0000 front 2024-05-03T19:03:37.951864+0000 (oldest deadline 2024-05-03T19:04:02.651644+0000) 2024-05-03T19:04:31.999 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:31 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:31.549+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-05-03T19:03:46.653109+0000 front 2024-05-03T19:03:46.653179+0000 (oldest deadline 2024-05-03T19:04:12.553001+0000) 2024-05-03T19:04:31.999 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:31 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:31.549+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.45:6812 osd.4 since back 2024-05-03T19:03:54.853856+0000 front 2024-05-03T19:03:54.853882+0000 (oldest deadline 2024-05-03T19:04:20.754013+0000) 2024-05-03T19:04:31.999 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:31 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:31.549+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.113:6812 osd.5 since back 2024-05-03T19:04:04.755716+0000 front 2024-05-03T19:04:04.755535+0000 (oldest deadline 2024-05-03T19:04:27.555916+0000) 2024-05-03T19:04:32.209 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:31 smithi032 systemd[1]: Stopping Ceph osd.9 for f500455e-097b-11ef-bc97-c7b262605968... 2024-05-03T19:04:32.209 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:32 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:04:32.047+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:23.927793+0000 front 2024-05-03T19:03:23.927723+0000 (oldest deadline 2024-05-03T19:03:46.227921+0000) 2024-05-03T19:04:32.210 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:32 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:04:32.047+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:29.729118+0000 front 2024-05-03T19:03:29.729178+0000 (oldest deadline 2024-05-03T19:03:54.429041+0000) 2024-05-03T19:04:32.210 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:32 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:04:32.047+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:40.231110+0000 front 2024-05-03T19:03:40.230916+0000 (oldest deadline 2024-05-03T19:04:03.130962+0000) 2024-05-03T19:04:32.210 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:32 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:04:32.047+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-05-03T19:03:46.032125+0000 front 2024-05-03T19:03:46.032053+0000 (oldest deadline 2024-05-03T19:04:11.332075+0000) 2024-05-03T19:04:32.210 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:32 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:04:32.047+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.113:6812 osd.5 since back 2024-05-03T19:04:03.034370+0000 front 2024-05-03T19:04:03.034324+0000 (oldest deadline 2024-05-03T19:04:28.334378+0000) 2024-05-03T19:04:32.210 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:32 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:04:32.208+0000 7fdaabc3c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-03T19:04:32.210 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:32 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:04:32.208+0000 7fdaabc3c700 -1 osd.9 85 *** Got signal Terminated *** 2024-05-03T19:04:32.210 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:32 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:04:32.208+0000 7fdaabc3c700 -1 osd.9 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-03T19:04:32.801 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:32 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:32.442+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:25.060144+0000 front 2024-05-03T19:03:25.060125+0000 (oldest deadline 2024-05-03T19:03:46.760321+0000) 2024-05-03T19:04:32.802 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:32 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:32.442+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:31.362284+0000 front 2024-05-03T19:03:31.362156+0000 (oldest deadline 2024-05-03T19:03:56.662340+0000) 2024-05-03T19:04:32.802 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:32 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:32.442+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:40.163402+0000 front 2024-05-03T19:03:40.163384+0000 (oldest deadline 2024-05-03T19:04:03.663534+0000) 2024-05-03T19:04:32.802 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:32 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:32.442+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.45:6812 osd.4 since back 2024-05-03T19:03:56.565512+0000 front 2024-05-03T19:03:56.565524+0000 (oldest deadline 2024-05-03T19:04:19.465417+0000) 2024-05-03T19:04:32.802 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:32 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:32.442+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.113:6812 osd.5 since back 2024-05-03T19:04:04.567504+0000 front 2024-05-03T19:04:04.567582+0000 (oldest deadline 2024-05-03T19:04:28.067641+0000) 2024-05-03T19:04:32.998 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:32 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:32.585+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:25.050521+0000 front 2024-05-03T19:03:25.050536+0000 (oldest deadline 2024-05-03T19:03:49.150320+0000) 2024-05-03T19:04:32.998 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:32 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:32.585+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:29.150598+0000 front 2024-05-03T19:03:29.150627+0000 (oldest deadline 2024-05-03T19:03:55.050741+0000) 2024-05-03T19:04:32.998 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:32 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:32.585+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:37.951720+0000 front 2024-05-03T19:03:37.951864+0000 (oldest deadline 2024-05-03T19:04:02.651644+0000) 2024-05-03T19:04:32.999 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:32 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:32.585+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-05-03T19:03:46.653109+0000 front 2024-05-03T19:03:46.653179+0000 (oldest deadline 2024-05-03T19:04:12.553001+0000) 2024-05-03T19:04:32.999 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:32 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:32.585+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.45:6812 osd.4 since back 2024-05-03T19:03:54.853856+0000 front 2024-05-03T19:03:54.853882+0000 (oldest deadline 2024-05-03T19:04:20.754013+0000) 2024-05-03T19:04:32.999 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:32 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:32.585+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.113:6812 osd.5 since back 2024-05-03T19:04:04.755716+0000 front 2024-05-03T19:04:04.755535+0000 (oldest deadline 2024-05-03T19:04:27.555916+0000) 2024-05-03T19:04:33.318 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:33 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:04:33.021+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:23.927793+0000 front 2024-05-03T19:03:23.927723+0000 (oldest deadline 2024-05-03T19:03:46.227921+0000) 2024-05-03T19:04:33.319 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:33 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:04:33.021+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:29.729118+0000 front 2024-05-03T19:03:29.729178+0000 (oldest deadline 2024-05-03T19:03:54.429041+0000) 2024-05-03T19:04:33.319 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:33 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:04:33.021+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:40.231110+0000 front 2024-05-03T19:03:40.230916+0000 (oldest deadline 2024-05-03T19:04:03.130962+0000) 2024-05-03T19:04:33.319 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:33 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:04:33.021+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-05-03T19:03:46.032125+0000 front 2024-05-03T19:03:46.032053+0000 (oldest deadline 2024-05-03T19:04:11.332075+0000) 2024-05-03T19:04:33.319 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:33 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:04:33.021+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.113:6812 osd.5 since back 2024-05-03T19:04:03.034370+0000 front 2024-05-03T19:04:03.034324+0000 (oldest deadline 2024-05-03T19:04:28.334378+0000) 2024-05-03T19:04:33.801 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:33 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:33.411+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:25.060144+0000 front 2024-05-03T19:03:25.060125+0000 (oldest deadline 2024-05-03T19:03:46.760321+0000) 2024-05-03T19:04:33.802 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:33 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:33.411+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:31.362284+0000 front 2024-05-03T19:03:31.362156+0000 (oldest deadline 2024-05-03T19:03:56.662340+0000) 2024-05-03T19:04:33.802 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:33 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:33.411+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:40.163402+0000 front 2024-05-03T19:03:40.163384+0000 (oldest deadline 2024-05-03T19:04:03.663534+0000) 2024-05-03T19:04:33.802 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:33 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:33.411+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.45:6812 osd.4 since back 2024-05-03T19:03:56.565512+0000 front 2024-05-03T19:03:56.565524+0000 (oldest deadline 2024-05-03T19:04:19.465417+0000) 2024-05-03T19:04:33.802 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:33 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:33.411+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.113:6812 osd.5 since back 2024-05-03T19:04:04.567504+0000 front 2024-05-03T19:04:04.567582+0000 (oldest deadline 2024-05-03T19:04:28.067641+0000) 2024-05-03T19:04:33.998 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:33 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:33.542+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:25.050521+0000 front 2024-05-03T19:03:25.050536+0000 (oldest deadline 2024-05-03T19:03:49.150320+0000) 2024-05-03T19:04:33.998 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:33 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:33.542+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:29.150598+0000 front 2024-05-03T19:03:29.150627+0000 (oldest deadline 2024-05-03T19:03:55.050741+0000) 2024-05-03T19:04:33.999 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:33 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:33.542+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:37.951720+0000 front 2024-05-03T19:03:37.951864+0000 (oldest deadline 2024-05-03T19:04:02.651644+0000) 2024-05-03T19:04:33.999 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:33 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:33.542+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-05-03T19:03:46.653109+0000 front 2024-05-03T19:03:46.653179+0000 (oldest deadline 2024-05-03T19:04:12.553001+0000) 2024-05-03T19:04:33.999 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:33 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:33.542+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.45:6812 osd.4 since back 2024-05-03T19:03:54.853856+0000 front 2024-05-03T19:03:54.853882+0000 (oldest deadline 2024-05-03T19:04:20.754013+0000) 2024-05-03T19:04:33.999 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:33 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:33.542+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.113:6812 osd.5 since back 2024-05-03T19:04:04.755716+0000 front 2024-05-03T19:04:04.755535+0000 (oldest deadline 2024-05-03T19:04:27.555916+0000) 2024-05-03T19:04:34.318 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:33 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:04:33.988+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:23.927793+0000 front 2024-05-03T19:03:23.927723+0000 (oldest deadline 2024-05-03T19:03:46.227921+0000) 2024-05-03T19:04:34.319 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:33 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:04:33.988+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:29.729118+0000 front 2024-05-03T19:03:29.729178+0000 (oldest deadline 2024-05-03T19:03:54.429041+0000) 2024-05-03T19:04:34.319 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:33 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:04:33.988+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:40.231110+0000 front 2024-05-03T19:03:40.230916+0000 (oldest deadline 2024-05-03T19:04:03.130962+0000) 2024-05-03T19:04:34.319 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:33 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:04:33.988+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-05-03T19:03:46.032125+0000 front 2024-05-03T19:03:46.032053+0000 (oldest deadline 2024-05-03T19:04:11.332075+0000) 2024-05-03T19:04:34.319 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:33 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:04:33.988+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.113:6812 osd.5 since back 2024-05-03T19:04:03.034370+0000 front 2024-05-03T19:04:03.034324+0000 (oldest deadline 2024-05-03T19:04:28.334378+0000) 2024-05-03T19:04:34.801 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:34 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:34.450+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:25.060144+0000 front 2024-05-03T19:03:25.060125+0000 (oldest deadline 2024-05-03T19:03:46.760321+0000) 2024-05-03T19:04:34.802 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:34 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:34.450+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:31.362284+0000 front 2024-05-03T19:03:31.362156+0000 (oldest deadline 2024-05-03T19:03:56.662340+0000) 2024-05-03T19:04:34.802 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:34 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:34.450+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:40.163402+0000 front 2024-05-03T19:03:40.163384+0000 (oldest deadline 2024-05-03T19:04:03.663534+0000) 2024-05-03T19:04:34.802 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:34 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:34.450+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.45:6812 osd.4 since back 2024-05-03T19:03:56.565512+0000 front 2024-05-03T19:03:56.565524+0000 (oldest deadline 2024-05-03T19:04:19.465417+0000) 2024-05-03T19:04:34.802 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:34 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:34.450+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.113:6812 osd.5 since back 2024-05-03T19:04:04.567504+0000 front 2024-05-03T19:04:04.567582+0000 (oldest deadline 2024-05-03T19:04:28.067641+0000) 2024-05-03T19:04:34.998 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:34 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:34.533+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:25.050521+0000 front 2024-05-03T19:03:25.050536+0000 (oldest deadline 2024-05-03T19:03:49.150320+0000) 2024-05-03T19:04:34.998 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:34 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:34.533+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:29.150598+0000 front 2024-05-03T19:03:29.150627+0000 (oldest deadline 2024-05-03T19:03:55.050741+0000) 2024-05-03T19:04:34.998 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:34 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:34.533+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:37.951720+0000 front 2024-05-03T19:03:37.951864+0000 (oldest deadline 2024-05-03T19:04:02.651644+0000) 2024-05-03T19:04:34.999 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:34 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:34.533+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-05-03T19:03:46.653109+0000 front 2024-05-03T19:03:46.653179+0000 (oldest deadline 2024-05-03T19:04:12.553001+0000) 2024-05-03T19:04:34.999 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:34 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:34.533+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.45:6812 osd.4 since back 2024-05-03T19:03:54.853856+0000 front 2024-05-03T19:03:54.853882+0000 (oldest deadline 2024-05-03T19:04:20.754013+0000) 2024-05-03T19:04:34.999 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:34 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:34.533+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.113:6812 osd.5 since back 2024-05-03T19:04:04.755716+0000 front 2024-05-03T19:04:04.755535+0000 (oldest deadline 2024-05-03T19:04:27.555916+0000) 2024-05-03T19:04:35.318 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:35 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:04:35.015+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:23.927793+0000 front 2024-05-03T19:03:23.927723+0000 (oldest deadline 2024-05-03T19:03:46.227921+0000) 2024-05-03T19:04:35.319 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:35 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:04:35.015+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:29.729118+0000 front 2024-05-03T19:03:29.729178+0000 (oldest deadline 2024-05-03T19:03:54.429041+0000) 2024-05-03T19:04:35.319 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:35 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:04:35.015+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:40.231110+0000 front 2024-05-03T19:03:40.230916+0000 (oldest deadline 2024-05-03T19:04:03.130962+0000) 2024-05-03T19:04:35.319 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:35 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:04:35.015+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-05-03T19:03:46.032125+0000 front 2024-05-03T19:03:46.032053+0000 (oldest deadline 2024-05-03T19:04:11.332075+0000) 2024-05-03T19:04:35.319 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:35 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:04:35.015+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.113:6812 osd.5 since back 2024-05-03T19:04:03.034370+0000 front 2024-05-03T19:04:03.034324+0000 (oldest deadline 2024-05-03T19:04:28.334378+0000) 2024-05-03T19:04:35.801 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:35 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:35.475+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:25.060144+0000 front 2024-05-03T19:03:25.060125+0000 (oldest deadline 2024-05-03T19:03:46.760321+0000) 2024-05-03T19:04:35.801 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:35 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:35.475+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:31.362284+0000 front 2024-05-03T19:03:31.362156+0000 (oldest deadline 2024-05-03T19:03:56.662340+0000) 2024-05-03T19:04:35.802 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:35 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:35.475+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:40.163402+0000 front 2024-05-03T19:03:40.163384+0000 (oldest deadline 2024-05-03T19:04:03.663534+0000) 2024-05-03T19:04:35.802 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:35 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:35.475+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.45:6812 osd.4 since back 2024-05-03T19:03:56.565512+0000 front 2024-05-03T19:03:56.565524+0000 (oldest deadline 2024-05-03T19:04:19.465417+0000) 2024-05-03T19:04:35.802 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:35 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:35.475+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.113:6812 osd.5 since back 2024-05-03T19:04:04.567504+0000 front 2024-05-03T19:04:04.567582+0000 (oldest deadline 2024-05-03T19:04:28.067641+0000) 2024-05-03T19:04:35.998 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:35 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:35.577+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:25.050521+0000 front 2024-05-03T19:03:25.050536+0000 (oldest deadline 2024-05-03T19:03:49.150320+0000) 2024-05-03T19:04:35.998 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:35 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:35.577+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:29.150598+0000 front 2024-05-03T19:03:29.150627+0000 (oldest deadline 2024-05-03T19:03:55.050741+0000) 2024-05-03T19:04:35.998 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:35 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:35.577+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:37.951720+0000 front 2024-05-03T19:03:37.951864+0000 (oldest deadline 2024-05-03T19:04:02.651644+0000) 2024-05-03T19:04:35.999 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:35 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:35.577+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-05-03T19:03:46.653109+0000 front 2024-05-03T19:03:46.653179+0000 (oldest deadline 2024-05-03T19:04:12.553001+0000) 2024-05-03T19:04:35.999 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:35 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:35.577+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.45:6812 osd.4 since back 2024-05-03T19:03:54.853856+0000 front 2024-05-03T19:03:54.853882+0000 (oldest deadline 2024-05-03T19:04:20.754013+0000) 2024-05-03T19:04:35.999 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:35 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:35.577+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.113:6812 osd.5 since back 2024-05-03T19:04:04.755716+0000 front 2024-05-03T19:04:04.755535+0000 (oldest deadline 2024-05-03T19:04:27.555916+0000) 2024-05-03T19:04:35.999 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:35 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:35.577+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.32:6822 osd.6 since back 2024-05-03T19:04:11.656784+0000 front 2024-05-03T19:04:11.656743+0000 (oldest deadline 2024-05-03T19:04:34.556774+0000) 2024-05-03T19:04:36.318 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:35 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:04:35.975+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:23.927793+0000 front 2024-05-03T19:03:23.927723+0000 (oldest deadline 2024-05-03T19:03:46.227921+0000) 2024-05-03T19:04:36.319 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:35 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:04:35.975+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:29.729118+0000 front 2024-05-03T19:03:29.729178+0000 (oldest deadline 2024-05-03T19:03:54.429041+0000) 2024-05-03T19:04:36.319 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:35 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:04:35.975+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:40.231110+0000 front 2024-05-03T19:03:40.230916+0000 (oldest deadline 2024-05-03T19:04:03.130962+0000) 2024-05-03T19:04:36.319 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:35 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:04:35.975+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-05-03T19:03:46.032125+0000 front 2024-05-03T19:03:46.032053+0000 (oldest deadline 2024-05-03T19:04:11.332075+0000) 2024-05-03T19:04:36.319 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:35 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:04:35.975+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.113:6812 osd.5 since back 2024-05-03T19:04:03.034370+0000 front 2024-05-03T19:04:03.034324+0000 (oldest deadline 2024-05-03T19:04:28.334378+0000) 2024-05-03T19:04:36.801 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:36 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:36.502+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:25.060144+0000 front 2024-05-03T19:03:25.060125+0000 (oldest deadline 2024-05-03T19:03:46.760321+0000) 2024-05-03T19:04:36.802 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:36 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:36.502+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:31.362284+0000 front 2024-05-03T19:03:31.362156+0000 (oldest deadline 2024-05-03T19:03:56.662340+0000) 2024-05-03T19:04:36.802 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:36 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:36.502+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:40.163402+0000 front 2024-05-03T19:03:40.163384+0000 (oldest deadline 2024-05-03T19:04:03.663534+0000) 2024-05-03T19:04:36.802 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:36 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:36.502+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.45:6812 osd.4 since back 2024-05-03T19:03:56.565512+0000 front 2024-05-03T19:03:56.565524+0000 (oldest deadline 2024-05-03T19:04:19.465417+0000) 2024-05-03T19:04:36.802 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:36 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:36.502+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.113:6812 osd.5 since back 2024-05-03T19:04:04.567504+0000 front 2024-05-03T19:04:04.567582+0000 (oldest deadline 2024-05-03T19:04:28.067641+0000) 2024-05-03T19:04:36.802 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:36 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:36.502+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.32:6822 osd.6 since back 2024-05-03T19:04:11.568749+0000 front 2024-05-03T19:04:11.568683+0000 (oldest deadline 2024-05-03T19:04:36.268596+0000) 2024-05-03T19:04:36.998 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:36 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:36.562+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:25.050521+0000 front 2024-05-03T19:03:25.050536+0000 (oldest deadline 2024-05-03T19:03:49.150320+0000) 2024-05-03T19:04:36.999 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:36 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:36.562+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:29.150598+0000 front 2024-05-03T19:03:29.150627+0000 (oldest deadline 2024-05-03T19:03:55.050741+0000) 2024-05-03T19:04:36.999 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:36 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:36.562+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:37.951720+0000 front 2024-05-03T19:03:37.951864+0000 (oldest deadline 2024-05-03T19:04:02.651644+0000) 2024-05-03T19:04:36.999 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:36 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:36.562+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-05-03T19:03:46.653109+0000 front 2024-05-03T19:03:46.653179+0000 (oldest deadline 2024-05-03T19:04:12.553001+0000) 2024-05-03T19:04:36.999 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:36 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:36.562+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.45:6812 osd.4 since back 2024-05-03T19:03:54.853856+0000 front 2024-05-03T19:03:54.853882+0000 (oldest deadline 2024-05-03T19:04:20.754013+0000) 2024-05-03T19:04:36.999 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:36 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:36.562+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.113:6812 osd.5 since back 2024-05-03T19:04:04.755716+0000 front 2024-05-03T19:04:04.755535+0000 (oldest deadline 2024-05-03T19:04:27.555916+0000) 2024-05-03T19:04:36.999 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:36 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:36.562+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.32:6822 osd.6 since back 2024-05-03T19:04:11.656784+0000 front 2024-05-03T19:04:11.656743+0000 (oldest deadline 2024-05-03T19:04:34.556774+0000) 2024-05-03T19:04:37.318 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:37 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:04:37.010+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:23.927793+0000 front 2024-05-03T19:03:23.927723+0000 (oldest deadline 2024-05-03T19:03:46.227921+0000) 2024-05-03T19:04:37.319 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:37 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:04:37.010+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:29.729118+0000 front 2024-05-03T19:03:29.729178+0000 (oldest deadline 2024-05-03T19:03:54.429041+0000) 2024-05-03T19:04:37.319 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:37 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:04:37.010+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:40.231110+0000 front 2024-05-03T19:03:40.230916+0000 (oldest deadline 2024-05-03T19:04:03.130962+0000) 2024-05-03T19:04:37.319 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:37 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:04:37.010+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-05-03T19:03:46.032125+0000 front 2024-05-03T19:03:46.032053+0000 (oldest deadline 2024-05-03T19:04:11.332075+0000) 2024-05-03T19:04:37.319 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:37 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9[62578]: 2024-05-03T19:04:37.010+0000 7fdaa8230700 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.113:6812 osd.5 since back 2024-05-03T19:04:03.034370+0000 front 2024-05-03T19:04:03.034324+0000 (oldest deadline 2024-05-03T19:04:28.334378+0000) 2024-05-03T19:04:37.802 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:37 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:37.464+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:25.060144+0000 front 2024-05-03T19:03:25.060125+0000 (oldest deadline 2024-05-03T19:03:46.760321+0000) 2024-05-03T19:04:37.802 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:37 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:37.464+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:31.362284+0000 front 2024-05-03T19:03:31.362156+0000 (oldest deadline 2024-05-03T19:03:56.662340+0000) 2024-05-03T19:04:37.802 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:37 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:37.464+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:40.163402+0000 front 2024-05-03T19:03:40.163384+0000 (oldest deadline 2024-05-03T19:04:03.663534+0000) 2024-05-03T19:04:37.802 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:37 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:37.464+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.45:6812 osd.4 since back 2024-05-03T19:03:56.565512+0000 front 2024-05-03T19:03:56.565524+0000 (oldest deadline 2024-05-03T19:04:19.465417+0000) 2024-05-03T19:04:37.802 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:37 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:37.464+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.113:6812 osd.5 since back 2024-05-03T19:04:04.567504+0000 front 2024-05-03T19:04:04.567582+0000 (oldest deadline 2024-05-03T19:04:28.067641+0000) 2024-05-03T19:04:37.802 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:37 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:37.464+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.32:6822 osd.6 since back 2024-05-03T19:04:11.568749+0000 front 2024-05-03T19:04:11.568683+0000 (oldest deadline 2024-05-03T19:04:36.268596+0000) 2024-05-03T19:04:37.955 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:37 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:37.538+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:25.050521+0000 front 2024-05-03T19:03:25.050536+0000 (oldest deadline 2024-05-03T19:03:49.150320+0000) 2024-05-03T19:04:37.955 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:37 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:37.538+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:29.150598+0000 front 2024-05-03T19:03:29.150627+0000 (oldest deadline 2024-05-03T19:03:55.050741+0000) 2024-05-03T19:04:37.956 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:37 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:37.538+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:37.951720+0000 front 2024-05-03T19:03:37.951864+0000 (oldest deadline 2024-05-03T19:04:02.651644+0000) 2024-05-03T19:04:37.956 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:37 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:37.538+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-05-03T19:03:46.653109+0000 front 2024-05-03T19:03:46.653179+0000 (oldest deadline 2024-05-03T19:04:12.553001+0000) 2024-05-03T19:04:37.956 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:37 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:37.538+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.45:6812 osd.4 since back 2024-05-03T19:03:54.853856+0000 front 2024-05-03T19:03:54.853882+0000 (oldest deadline 2024-05-03T19:04:20.754013+0000) 2024-05-03T19:04:37.956 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:37 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:37.538+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.113:6812 osd.5 since back 2024-05-03T19:04:04.755716+0000 front 2024-05-03T19:04:04.755535+0000 (oldest deadline 2024-05-03T19:04:27.555916+0000) 2024-05-03T19:04:37.956 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:37 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:37.538+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.32:6822 osd.6 since back 2024-05-03T19:04:11.656784+0000 front 2024-05-03T19:04:11.656743+0000 (oldest deadline 2024-05-03T19:04:34.556774+0000) 2024-05-03T19:04:38.035 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:37 smithi032 podman[88185]: 2024-05-03 19:04:37.683773703 +0000 UTC m=+5.576966697 container died 0649800c937ecb0b5b2a8569d550107dd80e2eba809a81320e839106ca792d24 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240430, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-03T19:04:38.317 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:38 smithi032 podman[88185]: 2024-05-03 19:04:38.124456389 +0000 UTC m=+6.017649380 container cleanup 0649800c937ecb0b5b2a8569d550107dd80e2eba809a81320e839106ca792d24 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240430, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240501.193033-9a047fe) 2024-05-03T19:04:38.318 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:38 smithi032 bash[88185]: ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9 2024-05-03T19:04:38.568 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:38 smithi032 podman[88201]: 2024-05-03 19:04:38.317141735 +0000 UTC m=+0.627419592 container remove 0649800c937ecb0b5b2a8569d550107dd80e2eba809a81320e839106ca792d24 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240430, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-05-03T19:04:38.801 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:38 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:38.454+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:25.060144+0000 front 2024-05-03T19:03:25.060125+0000 (oldest deadline 2024-05-03T19:03:46.760321+0000) 2024-05-03T19:04:38.802 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:38 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:38.454+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:31.362284+0000 front 2024-05-03T19:03:31.362156+0000 (oldest deadline 2024-05-03T19:03:56.662340+0000) 2024-05-03T19:04:38.802 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:38 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:38.454+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:40.163402+0000 front 2024-05-03T19:03:40.163384+0000 (oldest deadline 2024-05-03T19:04:03.663534+0000) 2024-05-03T19:04:38.802 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:38 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:38.454+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.45:6812 osd.4 since back 2024-05-03T19:03:56.565512+0000 front 2024-05-03T19:03:56.565524+0000 (oldest deadline 2024-05-03T19:04:19.465417+0000) 2024-05-03T19:04:38.802 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:38 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:38.454+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.113:6812 osd.5 since back 2024-05-03T19:04:04.567504+0000 front 2024-05-03T19:04:04.567582+0000 (oldest deadline 2024-05-03T19:04:28.067641+0000) 2024-05-03T19:04:38.802 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:38 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:38.454+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.32:6822 osd.6 since back 2024-05-03T19:04:11.568749+0000 front 2024-05-03T19:04:11.568683+0000 (oldest deadline 2024-05-03T19:04:36.268596+0000) 2024-05-03T19:04:38.998 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:38 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:38.531+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:25.050521+0000 front 2024-05-03T19:03:25.050536+0000 (oldest deadline 2024-05-03T19:03:49.150320+0000) 2024-05-03T19:04:38.999 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:38 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:38.531+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:29.150598+0000 front 2024-05-03T19:03:29.150627+0000 (oldest deadline 2024-05-03T19:03:55.050741+0000) 2024-05-03T19:04:38.999 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:38 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:38.531+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:37.951720+0000 front 2024-05-03T19:03:37.951864+0000 (oldest deadline 2024-05-03T19:04:02.651644+0000) 2024-05-03T19:04:38.999 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:38 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:38.531+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-05-03T19:03:46.653109+0000 front 2024-05-03T19:03:46.653179+0000 (oldest deadline 2024-05-03T19:04:12.553001+0000) 2024-05-03T19:04:38.999 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:38 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:38.531+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.45:6812 osd.4 since back 2024-05-03T19:03:54.853856+0000 front 2024-05-03T19:03:54.853882+0000 (oldest deadline 2024-05-03T19:04:20.754013+0000) 2024-05-03T19:04:38.999 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:38 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:38.531+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.113:6812 osd.5 since back 2024-05-03T19:04:04.755716+0000 front 2024-05-03T19:04:04.755535+0000 (oldest deadline 2024-05-03T19:04:27.555916+0000) 2024-05-03T19:04:38.999 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:38 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:38.531+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.32:6822 osd.6 since back 2024-05-03T19:04:11.656784+0000 front 2024-05-03T19:04:11.656743+0000 (oldest deadline 2024-05-03T19:04:34.556774+0000) 2024-05-03T19:04:39.062 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:38 smithi032 podman[88296]: 2024-05-03 19:04:38.656672416 +0000 UTC m=+0.018299586 image pull ee4d846202496ba6096593e8e8becc0700a5eb7ebd95536031187f5f337d4012 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd 2024-05-03T19:04:39.062 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:38 smithi032 podman[88296]: 2024-05-03 19:04:38.776346482 +0000 UTC m=+0.137973622 container create 005183fa8101d8f4dbcbf6eee3eb7b13553338deacbff8bcf4309123e112957f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9-deactivate, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240430) 2024-05-03T19:04:39.063 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:38 smithi032 podman[88296]: 2024-05-03 19:04:38.987287302 +0000 UTC m=+0.348914440 container init 005183fa8101d8f4dbcbf6eee3eb7b13553338deacbff8bcf4309123e112957f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9-deactivate, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240430, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-03T19:04:39.063 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:38 smithi032 podman[88296]: 2024-05-03 19:04:38.99054586 +0000 UTC m=+0.352173000 container start 005183fa8101d8f4dbcbf6eee3eb7b13553338deacbff8bcf4309123e112957f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240430, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True) 2024-05-03T19:04:39.063 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:39 smithi032 podman[88296]: 2024-05-03 19:04:39.061977743 +0000 UTC m=+0.423604890 container attach 005183fa8101d8f4dbcbf6eee3eb7b13553338deacbff8bcf4309123e112957f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9-deactivate, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240430, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-05-03T19:04:39.319 INFO:journalctl@ceph.osd.9.smithi032.stdout:May 03 19:04:39 smithi032 podman[88296]: 2024-05-03 19:04:39.292243442 +0000 UTC m=+0.653870593 container died 005183fa8101d8f4dbcbf6eee3eb7b13553338deacbff8bcf4309123e112957f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-9-deactivate, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240430, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-05-03T19:04:39.802 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:39 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:39.445+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:25.060144+0000 front 2024-05-03T19:03:25.060125+0000 (oldest deadline 2024-05-03T19:03:46.760321+0000) 2024-05-03T19:04:39.802 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:39 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:39.445+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:31.362284+0000 front 2024-05-03T19:03:31.362156+0000 (oldest deadline 2024-05-03T19:03:56.662340+0000) 2024-05-03T19:04:39.802 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:39 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:39.445+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:40.163402+0000 front 2024-05-03T19:03:40.163384+0000 (oldest deadline 2024-05-03T19:04:03.663534+0000) 2024-05-03T19:04:39.803 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:39 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:39.445+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.45:6812 osd.4 since back 2024-05-03T19:03:56.565512+0000 front 2024-05-03T19:03:56.565524+0000 (oldest deadline 2024-05-03T19:04:19.465417+0000) 2024-05-03T19:04:39.803 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:39 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:39.445+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.113:6812 osd.5 since back 2024-05-03T19:04:04.567504+0000 front 2024-05-03T19:04:04.567582+0000 (oldest deadline 2024-05-03T19:04:28.067641+0000) 2024-05-03T19:04:39.803 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:39 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:39.445+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.32:6822 osd.6 since back 2024-05-03T19:04:11.568749+0000 front 2024-05-03T19:04:11.568683+0000 (oldest deadline 2024-05-03T19:04:36.268596+0000) 2024-05-03T19:04:39.865 DEBUG:teuthology.orchestra.run.smithi032:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f500455e-097b-11ef-bc97-c7b262605968@osd.9.service' 2024-05-03T19:04:39.905 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-03T19:04:39.906 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-05-03T19:04:39.906 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-05-03T19:04:39.906 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ceph-f500455e-097b-11ef-bc97-c7b262605968@osd.10 2024-05-03T19:04:39.933 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:39 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:39.570+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:25.050521+0000 front 2024-05-03T19:03:25.050536+0000 (oldest deadline 2024-05-03T19:03:49.150320+0000) 2024-05-03T19:04:39.933 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:39 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:39.570+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:29.150598+0000 front 2024-05-03T19:03:29.150627+0000 (oldest deadline 2024-05-03T19:03:55.050741+0000) 2024-05-03T19:04:39.934 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:39 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:39.570+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:37.951720+0000 front 2024-05-03T19:03:37.951864+0000 (oldest deadline 2024-05-03T19:04:02.651644+0000) 2024-05-03T19:04:39.934 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:39 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:39.570+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-05-03T19:03:46.653109+0000 front 2024-05-03T19:03:46.653179+0000 (oldest deadline 2024-05-03T19:04:12.553001+0000) 2024-05-03T19:04:39.934 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:39 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:39.570+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.45:6812 osd.4 since back 2024-05-03T19:03:54.853856+0000 front 2024-05-03T19:03:54.853882+0000 (oldest deadline 2024-05-03T19:04:20.754013+0000) 2024-05-03T19:04:39.934 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:39 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:39.570+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.113:6812 osd.5 since back 2024-05-03T19:04:04.755716+0000 front 2024-05-03T19:04:04.755535+0000 (oldest deadline 2024-05-03T19:04:27.555916+0000) 2024-05-03T19:04:39.934 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:39 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:39.570+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.32:6822 osd.6 since back 2024-05-03T19:04:11.656784+0000 front 2024-05-03T19:04:11.656743+0000 (oldest deadline 2024-05-03T19:04:34.556774+0000) 2024-05-03T19:04:40.232 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:39 smithi045 systemd[1]: Stopping Ceph osd.10 for f500455e-097b-11ef-bc97-c7b262605968... 2024-05-03T19:04:40.498 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:40 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:40.231+0000 7fc74b232700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-03T19:04:40.498 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:40 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:40.231+0000 7fc74b232700 -1 osd.10 85 *** Got signal Terminated *** 2024-05-03T19:04:40.498 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:40 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:40.231+0000 7fc74b232700 -1 osd.10 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-03T19:04:40.801 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:40 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:40.469+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:25.060144+0000 front 2024-05-03T19:03:25.060125+0000 (oldest deadline 2024-05-03T19:03:46.760321+0000) 2024-05-03T19:04:40.802 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:40 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:40.469+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:31.362284+0000 front 2024-05-03T19:03:31.362156+0000 (oldest deadline 2024-05-03T19:03:56.662340+0000) 2024-05-03T19:04:40.802 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:40 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:40.469+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:40.163402+0000 front 2024-05-03T19:03:40.163384+0000 (oldest deadline 2024-05-03T19:04:03.663534+0000) 2024-05-03T19:04:40.802 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:40 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:40.469+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.45:6812 osd.4 since back 2024-05-03T19:03:56.565512+0000 front 2024-05-03T19:03:56.565524+0000 (oldest deadline 2024-05-03T19:04:19.465417+0000) 2024-05-03T19:04:40.802 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:40 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:40.469+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.113:6812 osd.5 since back 2024-05-03T19:04:04.567504+0000 front 2024-05-03T19:04:04.567582+0000 (oldest deadline 2024-05-03T19:04:28.067641+0000) 2024-05-03T19:04:40.803 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:40 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:40.469+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.32:6822 osd.6 since back 2024-05-03T19:04:11.568749+0000 front 2024-05-03T19:04:11.568683+0000 (oldest deadline 2024-05-03T19:04:36.268596+0000) 2024-05-03T19:04:40.999 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:40 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:40.523+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:25.050521+0000 front 2024-05-03T19:03:25.050536+0000 (oldest deadline 2024-05-03T19:03:49.150320+0000) 2024-05-03T19:04:40.999 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:40 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:40.523+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:29.150598+0000 front 2024-05-03T19:03:29.150627+0000 (oldest deadline 2024-05-03T19:03:55.050741+0000) 2024-05-03T19:04:40.999 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:40 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:40.523+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:37.951720+0000 front 2024-05-03T19:03:37.951864+0000 (oldest deadline 2024-05-03T19:04:02.651644+0000) 2024-05-03T19:04:40.999 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:40 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:40.523+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-05-03T19:03:46.653109+0000 front 2024-05-03T19:03:46.653179+0000 (oldest deadline 2024-05-03T19:04:12.553001+0000) 2024-05-03T19:04:41.000 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:40 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:40.523+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.45:6812 osd.4 since back 2024-05-03T19:03:54.853856+0000 front 2024-05-03T19:03:54.853882+0000 (oldest deadline 2024-05-03T19:04:20.754013+0000) 2024-05-03T19:04:41.000 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:40 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:40.523+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.113:6812 osd.5 since back 2024-05-03T19:04:04.755716+0000 front 2024-05-03T19:04:04.755535+0000 (oldest deadline 2024-05-03T19:04:27.555916+0000) 2024-05-03T19:04:41.000 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:40 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:40.523+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.32:6822 osd.6 since back 2024-05-03T19:04:11.656784+0000 front 2024-05-03T19:04:11.656743+0000 (oldest deadline 2024-05-03T19:04:34.556774+0000) 2024-05-03T19:04:41.748 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:41 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:41.483+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:25.050521+0000 front 2024-05-03T19:03:25.050536+0000 (oldest deadline 2024-05-03T19:03:49.150320+0000) 2024-05-03T19:04:41.749 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:41 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:41.483+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:29.150598+0000 front 2024-05-03T19:03:29.150627+0000 (oldest deadline 2024-05-03T19:03:55.050741+0000) 2024-05-03T19:04:41.749 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:41 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:41.483+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:37.951720+0000 front 2024-05-03T19:03:37.951864+0000 (oldest deadline 2024-05-03T19:04:02.651644+0000) 2024-05-03T19:04:41.749 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:41 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:41.483+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-05-03T19:03:46.653109+0000 front 2024-05-03T19:03:46.653179+0000 (oldest deadline 2024-05-03T19:04:12.553001+0000) 2024-05-03T19:04:41.749 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:41 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:41.483+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.45:6812 osd.4 since back 2024-05-03T19:03:54.853856+0000 front 2024-05-03T19:03:54.853882+0000 (oldest deadline 2024-05-03T19:04:20.754013+0000) 2024-05-03T19:04:41.749 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:41 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:41.483+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.113:6812 osd.5 since back 2024-05-03T19:04:04.755716+0000 front 2024-05-03T19:04:04.755535+0000 (oldest deadline 2024-05-03T19:04:27.555916+0000) 2024-05-03T19:04:41.749 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:41 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:41.483+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.32:6822 osd.6 since back 2024-05-03T19:04:11.656784+0000 front 2024-05-03T19:04:11.656743+0000 (oldest deadline 2024-05-03T19:04:34.556774+0000) 2024-05-03T19:04:41.801 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:41 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:41.440+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:25.060144+0000 front 2024-05-03T19:03:25.060125+0000 (oldest deadline 2024-05-03T19:03:46.760321+0000) 2024-05-03T19:04:41.802 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:41 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:41.440+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:31.362284+0000 front 2024-05-03T19:03:31.362156+0000 (oldest deadline 2024-05-03T19:03:56.662340+0000) 2024-05-03T19:04:41.802 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:41 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:41.440+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:40.163402+0000 front 2024-05-03T19:03:40.163384+0000 (oldest deadline 2024-05-03T19:04:03.663534+0000) 2024-05-03T19:04:41.802 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:41 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:41.440+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.45:6812 osd.4 since back 2024-05-03T19:03:56.565512+0000 front 2024-05-03T19:03:56.565524+0000 (oldest deadline 2024-05-03T19:04:19.465417+0000) 2024-05-03T19:04:41.802 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:41 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:41.440+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.113:6812 osd.5 since back 2024-05-03T19:04:04.567504+0000 front 2024-05-03T19:04:04.567582+0000 (oldest deadline 2024-05-03T19:04:28.067641+0000) 2024-05-03T19:04:41.802 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:41 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:41.440+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.32:6822 osd.6 since back 2024-05-03T19:04:11.568749+0000 front 2024-05-03T19:04:11.568683+0000 (oldest deadline 2024-05-03T19:04:36.268596+0000) 2024-05-03T19:04:42.748 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:42 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:42.471+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:25.050521+0000 front 2024-05-03T19:03:25.050536+0000 (oldest deadline 2024-05-03T19:03:49.150320+0000) 2024-05-03T19:04:42.748 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:42 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:42.471+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:29.150598+0000 front 2024-05-03T19:03:29.150627+0000 (oldest deadline 2024-05-03T19:03:55.050741+0000) 2024-05-03T19:04:42.748 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:42 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:42.471+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:37.951720+0000 front 2024-05-03T19:03:37.951864+0000 (oldest deadline 2024-05-03T19:04:02.651644+0000) 2024-05-03T19:04:42.749 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:42 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:42.471+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-05-03T19:03:46.653109+0000 front 2024-05-03T19:03:46.653179+0000 (oldest deadline 2024-05-03T19:04:12.553001+0000) 2024-05-03T19:04:42.749 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:42 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:42.471+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.45:6812 osd.4 since back 2024-05-03T19:03:54.853856+0000 front 2024-05-03T19:03:54.853882+0000 (oldest deadline 2024-05-03T19:04:20.754013+0000) 2024-05-03T19:04:42.749 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:42 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:42.471+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.113:6812 osd.5 since back 2024-05-03T19:04:04.755716+0000 front 2024-05-03T19:04:04.755535+0000 (oldest deadline 2024-05-03T19:04:27.555916+0000) 2024-05-03T19:04:42.749 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:42 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:42.471+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.32:6822 osd.6 since back 2024-05-03T19:04:11.656784+0000 front 2024-05-03T19:04:11.656743+0000 (oldest deadline 2024-05-03T19:04:34.556774+0000) 2024-05-03T19:04:42.801 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:42 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:42.390+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:25.060144+0000 front 2024-05-03T19:03:25.060125+0000 (oldest deadline 2024-05-03T19:03:46.760321+0000) 2024-05-03T19:04:42.802 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:42 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:42.390+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:31.362284+0000 front 2024-05-03T19:03:31.362156+0000 (oldest deadline 2024-05-03T19:03:56.662340+0000) 2024-05-03T19:04:42.802 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:42 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:42.390+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:40.163402+0000 front 2024-05-03T19:03:40.163384+0000 (oldest deadline 2024-05-03T19:04:03.663534+0000) 2024-05-03T19:04:42.802 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:42 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:42.390+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.45:6812 osd.4 since back 2024-05-03T19:03:56.565512+0000 front 2024-05-03T19:03:56.565524+0000 (oldest deadline 2024-05-03T19:04:19.465417+0000) 2024-05-03T19:04:42.802 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:42 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:42.390+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.113:6812 osd.5 since back 2024-05-03T19:04:04.567504+0000 front 2024-05-03T19:04:04.567582+0000 (oldest deadline 2024-05-03T19:04:28.067641+0000) 2024-05-03T19:04:42.802 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:42 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:42.390+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.32:6822 osd.6 since back 2024-05-03T19:04:11.568749+0000 front 2024-05-03T19:04:11.568683+0000 (oldest deadline 2024-05-03T19:04:36.268596+0000) 2024-05-03T19:04:42.803 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:42 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:42.390+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.45:6820 osd.7 since back 2024-05-03T19:04:16.269280+0000 front 2024-05-03T19:04:16.269213+0000 (oldest deadline 2024-05-03T19:04:42.169184+0000) 2024-05-03T19:04:43.802 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:43 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:43.354+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:25.060144+0000 front 2024-05-03T19:03:25.060125+0000 (oldest deadline 2024-05-03T19:03:46.760321+0000) 2024-05-03T19:04:43.802 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:43 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:43.354+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:31.362284+0000 front 2024-05-03T19:03:31.362156+0000 (oldest deadline 2024-05-03T19:03:56.662340+0000) 2024-05-03T19:04:43.802 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:43 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:43.354+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:40.163402+0000 front 2024-05-03T19:03:40.163384+0000 (oldest deadline 2024-05-03T19:04:03.663534+0000) 2024-05-03T19:04:43.802 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:43 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:43.354+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.45:6812 osd.4 since back 2024-05-03T19:03:56.565512+0000 front 2024-05-03T19:03:56.565524+0000 (oldest deadline 2024-05-03T19:04:19.465417+0000) 2024-05-03T19:04:43.802 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:43 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:43.354+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.113:6812 osd.5 since back 2024-05-03T19:04:04.567504+0000 front 2024-05-03T19:04:04.567582+0000 (oldest deadline 2024-05-03T19:04:28.067641+0000) 2024-05-03T19:04:43.803 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:43 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:43.354+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.32:6822 osd.6 since back 2024-05-03T19:04:11.568749+0000 front 2024-05-03T19:04:11.568683+0000 (oldest deadline 2024-05-03T19:04:36.268596+0000) 2024-05-03T19:04:43.803 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:43 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:43.354+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.45:6820 osd.7 since back 2024-05-03T19:04:16.269280+0000 front 2024-05-03T19:04:16.269213+0000 (oldest deadline 2024-05-03T19:04:42.169184+0000) 2024-05-03T19:04:43.998 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:43 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:43.505+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:25.050521+0000 front 2024-05-03T19:03:25.050536+0000 (oldest deadline 2024-05-03T19:03:49.150320+0000) 2024-05-03T19:04:43.998 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:43 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:43.505+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:29.150598+0000 front 2024-05-03T19:03:29.150627+0000 (oldest deadline 2024-05-03T19:03:55.050741+0000) 2024-05-03T19:04:43.998 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:43 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:43.505+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:37.951720+0000 front 2024-05-03T19:03:37.951864+0000 (oldest deadline 2024-05-03T19:04:02.651644+0000) 2024-05-03T19:04:43.999 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:43 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:43.505+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-05-03T19:03:46.653109+0000 front 2024-05-03T19:03:46.653179+0000 (oldest deadline 2024-05-03T19:04:12.553001+0000) 2024-05-03T19:04:43.999 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:43 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:43.505+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.45:6812 osd.4 since back 2024-05-03T19:03:54.853856+0000 front 2024-05-03T19:03:54.853882+0000 (oldest deadline 2024-05-03T19:04:20.754013+0000) 2024-05-03T19:04:43.999 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:43 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:43.505+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.113:6812 osd.5 since back 2024-05-03T19:04:04.755716+0000 front 2024-05-03T19:04:04.755535+0000 (oldest deadline 2024-05-03T19:04:27.555916+0000) 2024-05-03T19:04:43.999 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:43 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:43.505+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.32:6822 osd.6 since back 2024-05-03T19:04:11.656784+0000 front 2024-05-03T19:04:11.656743+0000 (oldest deadline 2024-05-03T19:04:34.556774+0000) 2024-05-03T19:04:44.801 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:44 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:44.366+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:25.060144+0000 front 2024-05-03T19:03:25.060125+0000 (oldest deadline 2024-05-03T19:03:46.760321+0000) 2024-05-03T19:04:44.802 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:44 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:44.366+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:31.362284+0000 front 2024-05-03T19:03:31.362156+0000 (oldest deadline 2024-05-03T19:03:56.662340+0000) 2024-05-03T19:04:44.802 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:44 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:44.366+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:40.163402+0000 front 2024-05-03T19:03:40.163384+0000 (oldest deadline 2024-05-03T19:04:03.663534+0000) 2024-05-03T19:04:44.802 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:44 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:44.366+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.45:6812 osd.4 since back 2024-05-03T19:03:56.565512+0000 front 2024-05-03T19:03:56.565524+0000 (oldest deadline 2024-05-03T19:04:19.465417+0000) 2024-05-03T19:04:44.802 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:44 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:44.366+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.113:6812 osd.5 since back 2024-05-03T19:04:04.567504+0000 front 2024-05-03T19:04:04.567582+0000 (oldest deadline 2024-05-03T19:04:28.067641+0000) 2024-05-03T19:04:44.802 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:44 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:44.366+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.32:6822 osd.6 since back 2024-05-03T19:04:11.568749+0000 front 2024-05-03T19:04:11.568683+0000 (oldest deadline 2024-05-03T19:04:36.268596+0000) 2024-05-03T19:04:44.802 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:44 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:44.366+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.45:6820 osd.7 since back 2024-05-03T19:04:16.269280+0000 front 2024-05-03T19:04:16.269213+0000 (oldest deadline 2024-05-03T19:04:42.169184+0000) 2024-05-03T19:04:44.998 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:44 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:44.528+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:25.050521+0000 front 2024-05-03T19:03:25.050536+0000 (oldest deadline 2024-05-03T19:03:49.150320+0000) 2024-05-03T19:04:44.998 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:44 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:44.528+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:29.150598+0000 front 2024-05-03T19:03:29.150627+0000 (oldest deadline 2024-05-03T19:03:55.050741+0000) 2024-05-03T19:04:44.999 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:44 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:44.528+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:37.951720+0000 front 2024-05-03T19:03:37.951864+0000 (oldest deadline 2024-05-03T19:04:02.651644+0000) 2024-05-03T19:04:44.999 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:44 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:44.528+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.32:6814 osd.3 since back 2024-05-03T19:03:46.653109+0000 front 2024-05-03T19:03:46.653179+0000 (oldest deadline 2024-05-03T19:04:12.553001+0000) 2024-05-03T19:04:44.999 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:44 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:44.528+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.45:6812 osd.4 since back 2024-05-03T19:03:54.853856+0000 front 2024-05-03T19:03:54.853882+0000 (oldest deadline 2024-05-03T19:04:20.754013+0000) 2024-05-03T19:04:44.999 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:44 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:44.528+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.113:6812 osd.5 since back 2024-05-03T19:04:04.755716+0000 front 2024-05-03T19:04:04.755535+0000 (oldest deadline 2024-05-03T19:04:27.555916+0000) 2024-05-03T19:04:44.999 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:44 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10[58291]: 2024-05-03T19:04:44.528+0000 7fc747826700 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.32:6822 osd.6 since back 2024-05-03T19:04:11.656784+0000 front 2024-05-03T19:04:11.656743+0000 (oldest deadline 2024-05-03T19:04:34.556774+0000) 2024-05-03T19:04:45.801 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:45 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:45.325+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:25.060144+0000 front 2024-05-03T19:03:25.060125+0000 (oldest deadline 2024-05-03T19:03:46.760321+0000) 2024-05-03T19:04:45.802 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:45 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:45.325+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:31.362284+0000 front 2024-05-03T19:03:31.362156+0000 (oldest deadline 2024-05-03T19:03:56.662340+0000) 2024-05-03T19:04:45.802 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:45 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:45.325+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:40.163402+0000 front 2024-05-03T19:03:40.163384+0000 (oldest deadline 2024-05-03T19:04:03.663534+0000) 2024-05-03T19:04:45.802 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:45 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:45.325+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.45:6812 osd.4 since back 2024-05-03T19:03:56.565512+0000 front 2024-05-03T19:03:56.565524+0000 (oldest deadline 2024-05-03T19:04:19.465417+0000) 2024-05-03T19:04:45.802 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:45 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:45.325+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.113:6812 osd.5 since back 2024-05-03T19:04:04.567504+0000 front 2024-05-03T19:04:04.567582+0000 (oldest deadline 2024-05-03T19:04:28.067641+0000) 2024-05-03T19:04:45.802 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:45 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:45.325+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.32:6822 osd.6 since back 2024-05-03T19:04:11.568749+0000 front 2024-05-03T19:04:11.568683+0000 (oldest deadline 2024-05-03T19:04:36.268596+0000) 2024-05-03T19:04:45.802 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:45 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:45.325+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.45:6820 osd.7 since back 2024-05-03T19:04:16.269280+0000 front 2024-05-03T19:04:16.269213+0000 (oldest deadline 2024-05-03T19:04:42.169184+0000) 2024-05-03T19:04:45.956 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:45 smithi045 podman[69143]: 2024-05-03 19:04:45.619549009 +0000 UTC m=+5.505147477 container died 81e9e188cd802ba39d23762829b4c54822b7611815010f2282efeb7d19308071 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240430, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-05-03T19:04:45.956 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:45 smithi045 podman[69143]: 2024-05-03 19:04:45.955461601 +0000 UTC m=+5.841060076 container cleanup 81e9e188cd802ba39d23762829b4c54822b7611815010f2282efeb7d19308071 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240430, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-05-03T19:04:46.248 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:45 smithi045 bash[69143]: ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10 2024-05-03T19:04:46.248 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:46 smithi045 podman[69160]: 2024-05-03 19:04:46.123251282 +0000 UTC m=+0.502706981 container remove 81e9e188cd802ba39d23762829b4c54822b7611815010f2282efeb7d19308071 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240430, org.label-schema.schema-version=1.0) 2024-05-03T19:04:46.801 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:46 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:46.316+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:25.060144+0000 front 2024-05-03T19:03:25.060125+0000 (oldest deadline 2024-05-03T19:03:46.760321+0000) 2024-05-03T19:04:46.802 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:46 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:46.316+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:31.362284+0000 front 2024-05-03T19:03:31.362156+0000 (oldest deadline 2024-05-03T19:03:56.662340+0000) 2024-05-03T19:04:46.802 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:46 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:46.316+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:40.163402+0000 front 2024-05-03T19:03:40.163384+0000 (oldest deadline 2024-05-03T19:04:03.663534+0000) 2024-05-03T19:04:46.802 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:46 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:46.316+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.45:6812 osd.4 since back 2024-05-03T19:03:56.565512+0000 front 2024-05-03T19:03:56.565524+0000 (oldest deadline 2024-05-03T19:04:19.465417+0000) 2024-05-03T19:04:46.802 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:46 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:46.316+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.113:6812 osd.5 since back 2024-05-03T19:04:04.567504+0000 front 2024-05-03T19:04:04.567582+0000 (oldest deadline 2024-05-03T19:04:28.067641+0000) 2024-05-03T19:04:46.803 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:46 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:46.316+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.32:6822 osd.6 since back 2024-05-03T19:04:11.568749+0000 front 2024-05-03T19:04:11.568683+0000 (oldest deadline 2024-05-03T19:04:36.268596+0000) 2024-05-03T19:04:46.803 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:46 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:46.316+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.45:6820 osd.7 since back 2024-05-03T19:04:16.269280+0000 front 2024-05-03T19:04:16.269213+0000 (oldest deadline 2024-05-03T19:04:42.169184+0000) 2024-05-03T19:04:46.869 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:46 smithi045 podman[69251]: 2024-05-03 19:04:46.540609951 +0000 UTC m=+0.115044536 container create 2eb662f042f18e66055a7ce727bc12017faad97453fbb5c1a55f58d33ea4c1f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10-deactivate, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240430, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe) 2024-05-03T19:04:46.869 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:46 smithi045 podman[69251]: 2024-05-03 19:04:46.444188099 +0000 UTC m=+0.018622669 image pull ee4d846202496ba6096593e8e8becc0700a5eb7ebd95536031187f5f337d4012 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd 2024-05-03T19:04:46.869 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:46 smithi045 podman[69251]: 2024-05-03 19:04:46.868055061 +0000 UTC m=+0.442489613 container init 2eb662f042f18e66055a7ce727bc12017faad97453fbb5c1a55f58d33ea4c1f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10-deactivate, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240430, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-05-03T19:04:47.173 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:46 smithi045 podman[69251]: 2024-05-03 19:04:46.871432467 +0000 UTC m=+0.445867025 container start 2eb662f042f18e66055a7ce727bc12017faad97453fbb5c1a55f58d33ea4c1f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240430, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-05-03T19:04:47.173 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:46 smithi045 podman[69251]: 2024-05-03 19:04:46.942852079 +0000 UTC m=+0.517286636 container attach 2eb662f042f18e66055a7ce727bc12017faad97453fbb5c1a55f58d33ea4c1f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10-deactivate, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240430, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-05-03T19:04:47.446 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:47 smithi045 podman[69251]: 2024-05-03 19:04:47.172961286 +0000 UTC m=+0.747395832 container died 2eb662f042f18e66055a7ce727bc12017faad97453fbb5c1a55f58d33ea4c1f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10-deactivate, ceph=True, org.label-schema.build-date=20240430, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-05-03T19:04:47.551 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:47 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:47.287+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:25.060144+0000 front 2024-05-03T19:03:25.060125+0000 (oldest deadline 2024-05-03T19:03:46.760321+0000) 2024-05-03T19:04:47.552 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:47 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:47.287+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:31.362284+0000 front 2024-05-03T19:03:31.362156+0000 (oldest deadline 2024-05-03T19:03:56.662340+0000) 2024-05-03T19:04:47.552 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:47 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:47.287+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:40.163402+0000 front 2024-05-03T19:03:40.163384+0000 (oldest deadline 2024-05-03T19:04:03.663534+0000) 2024-05-03T19:04:47.552 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:47 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:47.287+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.45:6812 osd.4 since back 2024-05-03T19:03:56.565512+0000 front 2024-05-03T19:03:56.565524+0000 (oldest deadline 2024-05-03T19:04:19.465417+0000) 2024-05-03T19:04:47.552 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:47 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:47.287+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.113:6812 osd.5 since back 2024-05-03T19:04:04.567504+0000 front 2024-05-03T19:04:04.567582+0000 (oldest deadline 2024-05-03T19:04:28.067641+0000) 2024-05-03T19:04:47.552 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:47 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:47.287+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.32:6822 osd.6 since back 2024-05-03T19:04:11.568749+0000 front 2024-05-03T19:04:11.568683+0000 (oldest deadline 2024-05-03T19:04:36.268596+0000) 2024-05-03T19:04:47.552 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:47 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:47.287+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.45:6820 osd.7 since back 2024-05-03T19:04:16.269280+0000 front 2024-05-03T19:04:16.269213+0000 (oldest deadline 2024-05-03T19:04:42.169184+0000) 2024-05-03T19:04:47.695 DEBUG:teuthology.orchestra.run.smithi045:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f500455e-097b-11ef-bc97-c7b262605968@osd.10.service' 2024-05-03T19:04:47.748 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:47 smithi045 podman[69277]: 2024-05-03 19:04:47.679326145 +0000 UTC m=+0.500815724 container remove 2eb662f042f18e66055a7ce727bc12017faad97453fbb5c1a55f58d33ea4c1f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-10-deactivate, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.build-date=20240430, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True) 2024-05-03T19:04:47.748 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:47 smithi045 systemd[1]: ceph-f500455e-097b-11ef-bc97-c7b262605968@osd.10.service: Deactivated successfully. 2024-05-03T19:04:47.748 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:47 smithi045 systemd[1]: Stopped Ceph osd.10 for f500455e-097b-11ef-bc97-c7b262605968. 2024-05-03T19:04:47.748 INFO:journalctl@ceph.osd.10.smithi045.stdout:May 03 19:04:47 smithi045 systemd[1]: ceph-f500455e-097b-11ef-bc97-c7b262605968@osd.10.service: Consumed 10.903s CPU time. 2024-05-03T19:04:47.775 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-03T19:04:47.776 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-05-03T19:04:47.776 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-05-03T19:04:47.776 DEBUG:teuthology.orchestra.run.smithi113:> sudo systemctl stop ceph-f500455e-097b-11ef-bc97-c7b262605968@osd.11 2024-05-03T19:04:48.051 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:47 smithi113 systemd[1]: Stopping Ceph osd.11 for f500455e-097b-11ef-bc97-c7b262605968... 2024-05-03T19:04:48.552 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:48 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:48.055+0000 7fe1bdc3d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-03T19:04:48.552 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:48 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:48.055+0000 7fe1bdc3d700 -1 osd.11 85 *** Got signal Terminated *** 2024-05-03T19:04:48.552 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:48 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:48.055+0000 7fe1bdc3d700 -1 osd.11 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-03T19:04:48.552 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:48 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:48.295+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:25.060144+0000 front 2024-05-03T19:03:25.060125+0000 (oldest deadline 2024-05-03T19:03:46.760321+0000) 2024-05-03T19:04:48.552 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:48 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:48.295+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:31.362284+0000 front 2024-05-03T19:03:31.362156+0000 (oldest deadline 2024-05-03T19:03:56.662340+0000) 2024-05-03T19:04:48.552 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:48 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:48.295+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:40.163402+0000 front 2024-05-03T19:03:40.163384+0000 (oldest deadline 2024-05-03T19:04:03.663534+0000) 2024-05-03T19:04:48.553 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:48 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:48.295+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.45:6812 osd.4 since back 2024-05-03T19:03:56.565512+0000 front 2024-05-03T19:03:56.565524+0000 (oldest deadline 2024-05-03T19:04:19.465417+0000) 2024-05-03T19:04:48.553 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:48 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:48.295+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.113:6812 osd.5 since back 2024-05-03T19:04:04.567504+0000 front 2024-05-03T19:04:04.567582+0000 (oldest deadline 2024-05-03T19:04:28.067641+0000) 2024-05-03T19:04:48.553 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:48 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:48.295+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.32:6822 osd.6 since back 2024-05-03T19:04:11.568749+0000 front 2024-05-03T19:04:11.568683+0000 (oldest deadline 2024-05-03T19:04:36.268596+0000) 2024-05-03T19:04:48.553 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:48 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:48.295+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.45:6820 osd.7 since back 2024-05-03T19:04:16.269280+0000 front 2024-05-03T19:04:16.269213+0000 (oldest deadline 2024-05-03T19:04:42.169184+0000) 2024-05-03T19:04:49.801 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:49 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:49.339+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:25.060144+0000 front 2024-05-03T19:03:25.060125+0000 (oldest deadline 2024-05-03T19:03:46.760321+0000) 2024-05-03T19:04:49.802 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:49 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:49.339+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:31.362284+0000 front 2024-05-03T19:03:31.362156+0000 (oldest deadline 2024-05-03T19:03:56.662340+0000) 2024-05-03T19:04:49.802 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:49 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:49.339+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:40.163402+0000 front 2024-05-03T19:03:40.163384+0000 (oldest deadline 2024-05-03T19:04:03.663534+0000) 2024-05-03T19:04:49.802 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:49 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:49.339+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.45:6812 osd.4 since back 2024-05-03T19:03:56.565512+0000 front 2024-05-03T19:03:56.565524+0000 (oldest deadline 2024-05-03T19:04:19.465417+0000) 2024-05-03T19:04:49.802 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:49 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:49.339+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.113:6812 osd.5 since back 2024-05-03T19:04:04.567504+0000 front 2024-05-03T19:04:04.567582+0000 (oldest deadline 2024-05-03T19:04:28.067641+0000) 2024-05-03T19:04:49.802 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:49 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:49.339+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.32:6822 osd.6 since back 2024-05-03T19:04:11.568749+0000 front 2024-05-03T19:04:11.568683+0000 (oldest deadline 2024-05-03T19:04:36.268596+0000) 2024-05-03T19:04:49.803 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:49 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:49.339+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.45:6820 osd.7 since back 2024-05-03T19:04:16.269280+0000 front 2024-05-03T19:04:16.269213+0000 (oldest deadline 2024-05-03T19:04:42.169184+0000) 2024-05-03T19:04:50.802 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:50 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:50.308+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:25.060144+0000 front 2024-05-03T19:03:25.060125+0000 (oldest deadline 2024-05-03T19:03:46.760321+0000) 2024-05-03T19:04:50.802 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:50 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:50.308+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:31.362284+0000 front 2024-05-03T19:03:31.362156+0000 (oldest deadline 2024-05-03T19:03:56.662340+0000) 2024-05-03T19:04:50.803 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:50 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:50.308+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:40.163402+0000 front 2024-05-03T19:03:40.163384+0000 (oldest deadline 2024-05-03T19:04:03.663534+0000) 2024-05-03T19:04:50.803 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:50 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:50.308+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.45:6812 osd.4 since back 2024-05-03T19:03:56.565512+0000 front 2024-05-03T19:03:56.565524+0000 (oldest deadline 2024-05-03T19:04:19.465417+0000) 2024-05-03T19:04:50.803 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:50 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:50.308+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.113:6812 osd.5 since back 2024-05-03T19:04:04.567504+0000 front 2024-05-03T19:04:04.567582+0000 (oldest deadline 2024-05-03T19:04:28.067641+0000) 2024-05-03T19:04:50.803 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:50 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:50.308+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.32:6822 osd.6 since back 2024-05-03T19:04:11.568749+0000 front 2024-05-03T19:04:11.568683+0000 (oldest deadline 2024-05-03T19:04:36.268596+0000) 2024-05-03T19:04:50.803 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:50 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:50.308+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.45:6820 osd.7 since back 2024-05-03T19:04:16.269280+0000 front 2024-05-03T19:04:16.269213+0000 (oldest deadline 2024-05-03T19:04:42.169184+0000) 2024-05-03T19:04:51.551 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:51 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:51.269+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:25.060144+0000 front 2024-05-03T19:03:25.060125+0000 (oldest deadline 2024-05-03T19:03:46.760321+0000) 2024-05-03T19:04:51.552 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:51 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:51.269+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:31.362284+0000 front 2024-05-03T19:03:31.362156+0000 (oldest deadline 2024-05-03T19:03:56.662340+0000) 2024-05-03T19:04:51.552 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:51 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:51.269+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:40.163402+0000 front 2024-05-03T19:03:40.163384+0000 (oldest deadline 2024-05-03T19:04:03.663534+0000) 2024-05-03T19:04:51.552 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:51 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:51.269+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.45:6812 osd.4 since back 2024-05-03T19:03:56.565512+0000 front 2024-05-03T19:03:56.565524+0000 (oldest deadline 2024-05-03T19:04:19.465417+0000) 2024-05-03T19:04:51.552 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:51 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:51.269+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.113:6812 osd.5 since back 2024-05-03T19:04:04.567504+0000 front 2024-05-03T19:04:04.567582+0000 (oldest deadline 2024-05-03T19:04:28.067641+0000) 2024-05-03T19:04:51.552 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:51 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:51.269+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.32:6822 osd.6 since back 2024-05-03T19:04:11.568749+0000 front 2024-05-03T19:04:11.568683+0000 (oldest deadline 2024-05-03T19:04:36.268596+0000) 2024-05-03T19:04:51.552 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:51 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:51.269+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.45:6820 osd.7 since back 2024-05-03T19:04:16.269280+0000 front 2024-05-03T19:04:16.269213+0000 (oldest deadline 2024-05-03T19:04:42.169184+0000) 2024-05-03T19:04:52.551 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:52 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:52.239+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.32:6806 osd.0 since back 2024-05-03T19:03:25.060144+0000 front 2024-05-03T19:03:25.060125+0000 (oldest deadline 2024-05-03T19:03:46.760321+0000) 2024-05-03T19:04:52.552 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:52 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:52.239+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-05-03T19:03:31.362284+0000 front 2024-05-03T19:03:31.362156+0000 (oldest deadline 2024-05-03T19:03:56.662340+0000) 2024-05-03T19:04:52.552 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:52 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:52.239+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.113:6804 osd.2 since back 2024-05-03T19:03:40.163402+0000 front 2024-05-03T19:03:40.163384+0000 (oldest deadline 2024-05-03T19:04:03.663534+0000) 2024-05-03T19:04:52.552 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:52 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:52.239+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.45:6812 osd.4 since back 2024-05-03T19:03:56.565512+0000 front 2024-05-03T19:03:56.565524+0000 (oldest deadline 2024-05-03T19:04:19.465417+0000) 2024-05-03T19:04:52.552 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:52 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:52.239+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.113:6812 osd.5 since back 2024-05-03T19:04:04.567504+0000 front 2024-05-03T19:04:04.567582+0000 (oldest deadline 2024-05-03T19:04:28.067641+0000) 2024-05-03T19:04:52.552 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:52 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:52.239+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.32:6822 osd.6 since back 2024-05-03T19:04:11.568749+0000 front 2024-05-03T19:04:11.568683+0000 (oldest deadline 2024-05-03T19:04:36.268596+0000) 2024-05-03T19:04:52.552 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:52 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11[59557]: 2024-05-03T19:04:52.239+0000 7fe1ba231700 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.45:6820 osd.7 since back 2024-05-03T19:04:16.269280+0000 front 2024-05-03T19:04:16.269213+0000 (oldest deadline 2024-05-03T19:04:42.169184+0000) 2024-05-03T19:04:53.691 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:53 smithi113 podman[70735]: 2024-05-03 19:04:53.403571051 +0000 UTC m=+5.436882900 container died 76ea97c5034c0caa13e8dcc33a0f9fe81ed3c2a1cff7e8a2837c9fb2af355556 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240430, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-05-03T19:04:53.692 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:53 smithi113 podman[70735]: 2024-05-03 19:04:53.690213101 +0000 UTC m=+5.723524942 container cleanup 76ea97c5034c0caa13e8dcc33a0f9fe81ed3c2a1cff7e8a2837c9fb2af355556 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11, org.label-schema.build-date=20240430, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe) 2024-05-03T19:04:54.051 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:53 smithi113 bash[70735]: ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11 2024-05-03T19:04:54.051 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:53 smithi113 podman[70752]: 2024-05-03 19:04:53.932875542 +0000 UTC m=+0.530448707 container remove 76ea97c5034c0caa13e8dcc33a0f9fe81ed3c2a1cff7e8a2837c9fb2af355556 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11, org.label-schema.build-date=20240430, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5) 2024-05-03T19:04:54.520 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:54 smithi113 podman[70841]: 2024-05-03 19:04:54.250435368 +0000 UTC m=+0.113874690 container create 3162a938388e2c1b9b7759b8ae4a47147c49380ad5fe6b4a3febd251a477385e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11-deactivate, org.label-schema.build-date=20240430, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True) 2024-05-03T19:04:54.520 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:54 smithi113 podman[70841]: 2024-05-03 19:04:54.154654747 +0000 UTC m=+0.018094079 image pull ee4d846202496ba6096593e8e8becc0700a5eb7ebd95536031187f5f337d4012 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd 2024-05-03T19:04:54.520 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:54 smithi113 podman[70841]: 2024-05-03 19:04:54.444459697 +0000 UTC m=+0.307899018 container init 3162a938388e2c1b9b7759b8ae4a47147c49380ad5fe6b4a3febd251a477385e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11-deactivate, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240430, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.license=GPLv2) 2024-05-03T19:04:54.520 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:54 smithi113 podman[70841]: 2024-05-03 19:04:54.448450578 +0000 UTC m=+0.311889911 container start 3162a938388e2c1b9b7759b8ae4a47147c49380ad5fe6b4a3febd251a477385e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11-deactivate, ceph=True, org.label-schema.build-date=20240430, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-05-03T19:04:54.520 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:54 smithi113 podman[70841]: 2024-05-03 19:04:54.519256629 +0000 UTC m=+0.382695960 container attach 3162a938388e2c1b9b7759b8ae4a47147c49380ad5fe6b4a3febd251a477385e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11-deactivate, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.build-date=20240430, org.label-schema.license=GPLv2) 2024-05-03T19:04:54.801 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:54 smithi113 podman[70841]: 2024-05-03 19:04:54.743438908 +0000 UTC m=+0.606878235 container died 3162a938388e2c1b9b7759b8ae4a47147c49380ad5fe6b4a3febd251a477385e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11-deactivate, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240430, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-03T19:04:55.288 DEBUG:teuthology.orchestra.run.smithi113:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f500455e-097b-11ef-bc97-c7b262605968@osd.11.service' 2024-05-03T19:04:55.301 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:55 smithi113 podman[70867]: 2024-05-03 19:04:55.264280206 +0000 UTC m=+0.510186069 container remove 3162a938388e2c1b9b7759b8ae4a47147c49380ad5fe6b4a3febd251a477385e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-osd-11-deactivate, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240430, ceph=True, org.label-schema.license=GPLv2) 2024-05-03T19:04:55.301 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:55 smithi113 systemd[1]: ceph-f500455e-097b-11ef-bc97-c7b262605968@osd.11.service: Deactivated successfully. 2024-05-03T19:04:55.301 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:55 smithi113 systemd[1]: Stopped Ceph osd.11 for f500455e-097b-11ef-bc97-c7b262605968. 2024-05-03T19:04:55.302 INFO:journalctl@ceph.osd.11.smithi113.stdout:May 03 19:04:55 smithi113 systemd[1]: ceph-f500455e-097b-11ef-bc97-c7b262605968@osd.11.service: Consumed 7.820s CPU time. 2024-05-03T19:04:55.324 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-03T19:04:55.324 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-05-03T19:04:55.325 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-05-03T19:04:55.325 DEBUG:teuthology.orchestra.run.smithi032:> sudo systemctl stop ceph-f500455e-097b-11ef-bc97-c7b262605968@mds.a 2024-05-03T19:04:55.618 INFO:journalctl@ceph.mds.a.smithi032.stdout:May 03 19:04:55 smithi032 systemd[1]: Stopping Ceph mds.a for f500455e-097b-11ef-bc97-c7b262605968... 2024-05-03T19:04:56.068 INFO:journalctl@ceph.mds.a.smithi032.stdout:May 03 19:04:55 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-mds-a[69444]: 2024-05-03T19:04:55.616+0000 7fa7226b6700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-03T19:04:56.068 INFO:journalctl@ceph.mds.a.smithi032.stdout:May 03 19:04:55 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-mds-a[69444]: 2024-05-03T19:04:55.616+0000 7fa7226b6700 -1 mds.a *** got signal Terminated *** 2024-05-03T19:04:58.951 INFO:journalctl@ceph.mds.a.smithi032.stdout:May 03 19:04:58 smithi032 podman[88404]: 2024-05-03 19:04:58.688162564 +0000 UTC m=+3.167377720 container died 3a5ed69a830240357db8fb389c47dd24041acc5681b0ffb0a82847d8f074e0e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-mds-a, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240430, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-03T19:04:59.234 INFO:journalctl@ceph.mds.a.smithi032.stdout:May 03 19:04:59 smithi032 podman[88404]: 2024-05-03 19:04:59.015807575 +0000 UTC m=+3.495022740 container cleanup 3a5ed69a830240357db8fb389c47dd24041acc5681b0ffb0a82847d8f074e0e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-mds-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240430, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.vendor=CentOS) 2024-05-03T19:04:59.235 INFO:journalctl@ceph.mds.a.smithi032.stdout:May 03 19:04:59 smithi032 bash[88404]: ceph-f500455e-097b-11ef-bc97-c7b262605968-mds-a 2024-05-03T19:04:59.449 DEBUG:teuthology.orchestra.run.smithi032:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f500455e-097b-11ef-bc97-c7b262605968@mds.a.service' 2024-05-03T19:04:59.521 INFO:journalctl@ceph.mds.a.smithi032.stdout:May 03 19:04:59 smithi032 podman[88418]: 2024-05-03 19:04:59.233502536 +0000 UTC m=+0.547932053 container remove 3a5ed69a830240357db8fb389c47dd24041acc5681b0ffb0a82847d8f074e0e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-mds-a, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240430, org.label-schema.schema-version=1.0) 2024-05-03T19:04:59.522 INFO:journalctl@ceph.mds.a.smithi032.stdout:May 03 19:04:59 smithi032 systemd[1]: ceph-f500455e-097b-11ef-bc97-c7b262605968@mds.a.service: Deactivated successfully. 2024-05-03T19:04:59.522 INFO:journalctl@ceph.mds.a.smithi032.stdout:May 03 19:04:59 smithi032 systemd[1]: Stopped Ceph mds.a for f500455e-097b-11ef-bc97-c7b262605968. 2024-05-03T19:04:59.522 INFO:journalctl@ceph.mds.a.smithi032.stdout:May 03 19:04:59 smithi032 systemd[1]: ceph-f500455e-097b-11ef-bc97-c7b262605968@mds.a.service: Consumed 1.172s CPU time. 2024-05-03T19:04:59.535 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-03T19:04:59.535 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-05-03T19:04:59.535 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-05-03T19:04:59.536 DEBUG:teuthology.orchestra.run.smithi032:> sudo systemctl stop ceph-f500455e-097b-11ef-bc97-c7b262605968@mds.d 2024-05-03T19:04:59.819 INFO:journalctl@ceph.mds.d.smithi032.stdout:May 03 19:04:59 smithi032 systemd[1]: Stopping Ceph mds.d for f500455e-097b-11ef-bc97-c7b262605968... 2024-05-03T19:05:00.319 INFO:journalctl@ceph.mds.d.smithi032.stdout:May 03 19:04:59 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-mds-d[68182]: 2024-05-03T19:04:59.860+0000 7fa682631700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-03T19:05:00.319 INFO:journalctl@ceph.mds.d.smithi032.stdout:May 03 19:04:59 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-mds-d[68182]: 2024-05-03T19:04:59.860+0000 7fa682631700 -1 mds.d *** got signal Terminated *** 2024-05-03T19:05:02.703 INFO:journalctl@ceph.mds.d.smithi032.stdout:May 03 19:05:02 smithi032 podman[88526]: 2024-05-03 19:05:02.378944388 +0000 UTC m=+2.610384020 container died 513c8de5cd19e0d4eba5896112d1b2eab056a5c07fb5f7729308912ac4e5f114 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-mds-d, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240430, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-05-03T19:05:02.703 INFO:journalctl@ceph.mds.d.smithi032.stdout:May 03 19:05:02 smithi032 podman[88526]: 2024-05-03 19:05:02.702646111 +0000 UTC m=+2.934085730 container cleanup 513c8de5cd19e0d4eba5896112d1b2eab056a5c07fb5f7729308912ac4e5f114 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-mds-d, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240430, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-05-03T19:05:03.069 INFO:journalctl@ceph.mds.d.smithi032.stdout:May 03 19:05:02 smithi032 bash[88526]: ceph-f500455e-097b-11ef-bc97-c7b262605968-mds-d 2024-05-03T19:05:03.070 INFO:journalctl@ceph.mds.d.smithi032.stdout:May 03 19:05:02 smithi032 podman[88540]: 2024-05-03 19:05:02.887096121 +0000 UTC m=+0.504180281 container remove 513c8de5cd19e0d4eba5896112d1b2eab056a5c07fb5f7729308912ac4e5f114 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-mds-d, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240430, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-05-03T19:05:03.115 DEBUG:teuthology.orchestra.run.smithi032:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f500455e-097b-11ef-bc97-c7b262605968@mds.d.service' 2024-05-03T19:05:03.147 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-03T19:05:03.147 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-05-03T19:05:03.148 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-05-03T19:05:03.148 DEBUG:teuthology.orchestra.run.smithi032:> sudo systemctl stop ceph-f500455e-097b-11ef-bc97-c7b262605968@mds.g 2024-05-03T19:05:03.457 INFO:journalctl@ceph.mds.g.smithi032.stdout:May 03 19:05:03 smithi032 systemd[1]: Stopping Ceph mds.g for f500455e-097b-11ef-bc97-c7b262605968... 2024-05-03T19:05:03.819 INFO:journalctl@ceph.mds.g.smithi032.stdout:May 03 19:05:03 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-mds-g[68981]: 2024-05-03T19:05:03.455+0000 7f67b43fe700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-03T19:05:03.819 INFO:journalctl@ceph.mds.g.smithi032.stdout:May 03 19:05:03 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-mds-g[68981]: 2024-05-03T19:05:03.455+0000 7f67b43fe700 -1 mds.g *** got signal Terminated *** 2024-05-03T19:05:07.157 INFO:journalctl@ceph.mds.g.smithi032.stdout:May 03 19:05:06 smithi032 podman[88645]: 2024-05-03 19:05:06.877251111 +0000 UTC m=+3.510404997 container died 43d6b3ac5db50f25bac20cb6d05a53bd6d45588387b4d51e26f2928c7e9c4455 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-mds-g, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240430, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-05-03T19:05:07.407 INFO:journalctl@ceph.mds.g.smithi032.stdout:May 03 19:05:07 smithi032 podman[88645]: 2024-05-03 19:05:07.247545531 +0000 UTC m=+3.880699416 container cleanup 43d6b3ac5db50f25bac20cb6d05a53bd6d45588387b4d51e26f2928c7e9c4455 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-mds-g, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.build-date=20240430, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-05-03T19:05:07.408 INFO:journalctl@ceph.mds.g.smithi032.stdout:May 03 19:05:07 smithi032 bash[88645]: ceph-f500455e-097b-11ef-bc97-c7b262605968-mds-g 2024-05-03T19:05:07.638 DEBUG:teuthology.orchestra.run.smithi032:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f500455e-097b-11ef-bc97-c7b262605968@mds.g.service' 2024-05-03T19:05:07.660 INFO:journalctl@ceph.mds.g.smithi032.stdout:May 03 19:05:07 smithi032 podman[88658]: 2024-05-03 19:05:07.406933687 +0000 UTC m=+0.527753900 container remove 43d6b3ac5db50f25bac20cb6d05a53bd6d45588387b4d51e26f2928c7e9c4455 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-mds-g, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.build-date=20240430) 2024-05-03T19:05:07.660 INFO:journalctl@ceph.mds.g.smithi032.stdout:May 03 19:05:07 smithi032 systemd[1]: ceph-f500455e-097b-11ef-bc97-c7b262605968@mds.g.service: Deactivated successfully. 2024-05-03T19:05:07.660 INFO:journalctl@ceph.mds.g.smithi032.stdout:May 03 19:05:07 smithi032 systemd[1]: Stopped Ceph mds.g for f500455e-097b-11ef-bc97-c7b262605968. 2024-05-03T19:05:07.660 INFO:journalctl@ceph.mds.g.smithi032.stdout:May 03 19:05:07 smithi032 systemd[1]: ceph-f500455e-097b-11ef-bc97-c7b262605968@mds.g.service: Consumed 1.166s CPU time. 2024-05-03T19:05:07.671 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-03T19:05:07.672 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-05-03T19:05:07.672 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-05-03T19:05:07.672 DEBUG:teuthology.orchestra.run.smithi032:> sudo systemctl stop ceph-f500455e-097b-11ef-bc97-c7b262605968@mds.j 2024-05-03T19:05:08.002 INFO:journalctl@ceph.mds.j.smithi032.stdout:May 03 19:05:07 smithi032 systemd[1]: Stopping Ceph mds.j for f500455e-097b-11ef-bc97-c7b262605968... 2024-05-03T19:05:08.319 INFO:journalctl@ceph.mds.j.smithi032.stdout:May 03 19:05:08 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-mds-j[67714]: 2024-05-03T19:05:08.000+0000 7efd1b415700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-03T19:05:08.319 INFO:journalctl@ceph.mds.j.smithi032.stdout:May 03 19:05:08 smithi032 ceph-f500455e-097b-11ef-bc97-c7b262605968-mds-j[67714]: 2024-05-03T19:05:08.000+0000 7efd1b415700 -1 mds.j *** got signal Terminated *** 2024-05-03T19:05:10.318 INFO:journalctl@ceph.mds.j.smithi032.stdout:May 03 19:05:10 smithi032 podman[88764]: 2024-05-03 19:05:10.070595045 +0000 UTC m=+2.177445390 container died 8bfc26b180308fd5a31275efad80c39c245d6bfe201dd92b5d4a041c864660f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-mds-j, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240430, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, ceph=True) 2024-05-03T19:05:10.677 INFO:journalctl@ceph.mds.j.smithi032.stdout:May 03 19:05:10 smithi032 podman[88764]: 2024-05-03 19:05:10.459372116 +0000 UTC m=+2.566222457 container cleanup 8bfc26b180308fd5a31275efad80c39c245d6bfe201dd92b5d4a041c864660f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-mds-j, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240430, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-05-03T19:05:10.678 INFO:journalctl@ceph.mds.j.smithi032.stdout:May 03 19:05:10 smithi032 bash[88764]: ceph-f500455e-097b-11ef-bc97-c7b262605968-mds-j 2024-05-03T19:05:10.804 DEBUG:teuthology.orchestra.run.smithi032:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f500455e-097b-11ef-bc97-c7b262605968@mds.j.service' 2024-05-03T19:05:10.837 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-03T19:05:10.838 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-05-03T19:05:10.838 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-05-03T19:05:10.838 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ceph-f500455e-097b-11ef-bc97-c7b262605968@mds.b 2024-05-03T19:05:11.156 INFO:journalctl@ceph.mds.b.smithi045.stdout:May 03 19:05:10 smithi045 systemd[1]: Stopping Ceph mds.b for f500455e-097b-11ef-bc97-c7b262605968... 2024-05-03T19:05:11.434 INFO:journalctl@ceph.mds.b.smithi045.stdout:May 03 19:05:11 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-mds-b[63790]: 2024-05-03T19:05:11.154+0000 7f737082f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-03T19:05:11.435 INFO:journalctl@ceph.mds.b.smithi045.stdout:May 03 19:05:11 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-mds-b[63790]: 2024-05-03T19:05:11.154+0000 7f737082f700 -1 mds.b *** got signal Terminated *** 2024-05-03T19:05:14.440 INFO:journalctl@ceph.mds.b.smithi045.stdout:May 03 19:05:14 smithi045 podman[69355]: 2024-05-03 19:05:14.139358207 +0000 UTC m=+3.107462219 container died f3935e6c79c24fc8bd260414749603f62b715186361832ee0121efba5090ff44 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-mds-b, org.label-schema.build-date=20240430, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-05-03T19:05:14.748 INFO:journalctl@ceph.mds.b.smithi045.stdout:May 03 19:05:14 smithi045 podman[69355]: 2024-05-03 19:05:14.537977832 +0000 UTC m=+3.506081892 container cleanup f3935e6c79c24fc8bd260414749603f62b715186361832ee0121efba5090ff44 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-mds-b, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240430, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-05-03T19:05:14.749 INFO:journalctl@ceph.mds.b.smithi045.stdout:May 03 19:05:14 smithi045 bash[69355]: ceph-f500455e-097b-11ef-bc97-c7b262605968-mds-b 2024-05-03T19:05:14.922 DEBUG:teuthology.orchestra.run.smithi045:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f500455e-097b-11ef-bc97-c7b262605968@mds.b.service' 2024-05-03T19:05:14.958 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-03T19:05:14.958 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-05-03T19:05:14.959 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-05-03T19:05:14.959 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ceph-f500455e-097b-11ef-bc97-c7b262605968@mds.e 2024-05-03T19:05:15.498 INFO:journalctl@ceph.mds.e.smithi045.stdout:May 03 19:05:15 smithi045 systemd[1]: Stopping Ceph mds.e for f500455e-097b-11ef-bc97-c7b262605968... 2024-05-03T19:05:15.499 INFO:journalctl@ceph.mds.e.smithi045.stdout:May 03 19:05:15 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-mds-e[63167]: 2024-05-03T19:05:15.274+0000 7f6ea0bde700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-03T19:05:15.499 INFO:journalctl@ceph.mds.e.smithi045.stdout:May 03 19:05:15 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-mds-e[63167]: 2024-05-03T19:05:15.274+0000 7f6ea0bde700 -1 mds.e *** got signal Terminated *** 2024-05-03T19:05:17.218 INFO:journalctl@ceph.mds.e.smithi045.stdout:May 03 19:05:16 smithi045 podman[69490]: 2024-05-03 19:05:16.964656744 +0000 UTC m=+1.778743159 container died 5eec80064e6eb87f440e59a997bb707af8b0e384603302a32628d96f766a582d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-mds-e, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240430, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux ) 2024-05-03T19:05:17.498 INFO:journalctl@ceph.mds.e.smithi045.stdout:May 03 19:05:17 smithi045 podman[69490]: 2024-05-03 19:05:17.291561683 +0000 UTC m=+2.105648102 container cleanup 5eec80064e6eb87f440e59a997bb707af8b0e384603302a32628d96f766a582d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-mds-e, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240430, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-03T19:05:17.498 INFO:journalctl@ceph.mds.e.smithi045.stdout:May 03 19:05:17 smithi045 bash[69490]: ceph-f500455e-097b-11ef-bc97-c7b262605968-mds-e 2024-05-03T19:05:17.728 DEBUG:teuthology.orchestra.run.smithi045:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f500455e-097b-11ef-bc97-c7b262605968@mds.e.service' 2024-05-03T19:05:17.762 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-03T19:05:17.762 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-05-03T19:05:17.763 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-05-03T19:05:17.763 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ceph-f500455e-097b-11ef-bc97-c7b262605968@mds.h 2024-05-03T19:05:18.096 INFO:journalctl@ceph.mds.h.smithi045.stdout:May 03 19:05:17 smithi045 systemd[1]: Stopping Ceph mds.h for f500455e-097b-11ef-bc97-c7b262605968... 2024-05-03T19:05:18.498 INFO:journalctl@ceph.mds.h.smithi045.stdout:May 03 19:05:18 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-mds-h[63473]: 2024-05-03T19:05:18.094+0000 7fa1a2011700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-03T19:05:18.498 INFO:journalctl@ceph.mds.h.smithi045.stdout:May 03 19:05:18 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-mds-h[63473]: 2024-05-03T19:05:18.094+0000 7fa1a2011700 -1 mds.h *** got signal Terminated *** 2024-05-03T19:05:22.248 INFO:journalctl@ceph.mds.h.smithi045.stdout:May 03 19:05:22 smithi045 podman[69611]: 2024-05-03 19:05:22.10805809 +0000 UTC m=+4.113670155 container died 7872d57d7bb32c444723390a305fe3c8c56a4010b5b29e42484396de703442c5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-mds-h, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.build-date=20240430, GIT_CLEAN=True, ceph=True) 2024-05-03T19:05:22.748 INFO:journalctl@ceph.mds.h.smithi045.stdout:May 03 19:05:22 smithi045 podman[69611]: 2024-05-03 19:05:22.578176482 +0000 UTC m=+4.583788546 container cleanup 7872d57d7bb32c444723390a305fe3c8c56a4010b5b29e42484396de703442c5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-mds-h, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240430, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-05-03T19:05:22.748 INFO:journalctl@ceph.mds.h.smithi045.stdout:May 03 19:05:22 smithi045 bash[69611]: ceph-f500455e-097b-11ef-bc97-c7b262605968-mds-h 2024-05-03T19:05:23.011 DEBUG:teuthology.orchestra.run.smithi045:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f500455e-097b-11ef-bc97-c7b262605968@mds.h.service' 2024-05-03T19:05:23.043 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-03T19:05:23.043 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-05-03T19:05:23.043 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-05-03T19:05:23.043 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ceph-f500455e-097b-11ef-bc97-c7b262605968@mds.k 2024-05-03T19:05:23.374 INFO:journalctl@ceph.mds.k.smithi045.stdout:May 03 19:05:23 smithi045 systemd[1]: Stopping Ceph mds.k for f500455e-097b-11ef-bc97-c7b262605968... 2024-05-03T19:05:23.748 INFO:journalctl@ceph.mds.k.smithi045.stdout:May 03 19:05:23 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-mds-k[62849]: 2024-05-03T19:05:23.372+0000 7f1e1d207700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-03T19:05:23.748 INFO:journalctl@ceph.mds.k.smithi045.stdout:May 03 19:05:23 smithi045 ceph-f500455e-097b-11ef-bc97-c7b262605968-mds-k[62849]: 2024-05-03T19:05:23.372+0000 7f1e1d207700 -1 mds.k *** got signal Terminated *** 2024-05-03T19:05:25.748 INFO:journalctl@ceph.mds.k.smithi045.stdout:May 03 19:05:25 smithi045 podman[69732]: 2024-05-03 19:05:25.486641873 +0000 UTC m=+2.222659644 container died 01184527005bcc1c52bab3de064db9e891de3a51c00c75bcaf192db2f689a34e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-mds-k, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240430, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-05-03T19:05:26.192 INFO:journalctl@ceph.mds.k.smithi045.stdout:May 03 19:05:25 smithi045 podman[69732]: 2024-05-03 19:05:25.823487864 +0000 UTC m=+2.559505633 container cleanup 01184527005bcc1c52bab3de064db9e891de3a51c00c75bcaf192db2f689a34e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-mds-k, org.label-schema.build-date=20240430, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-05-03T19:05:26.193 INFO:journalctl@ceph.mds.k.smithi045.stdout:May 03 19:05:25 smithi045 bash[69732]: ceph-f500455e-097b-11ef-bc97-c7b262605968-mds-k 2024-05-03T19:05:26.193 INFO:journalctl@ceph.mds.k.smithi045.stdout:May 03 19:05:25 smithi045 podman[69746]: 2024-05-03 19:05:25.991026465 +0000 UTC m=+0.503233861 container remove 01184527005bcc1c52bab3de064db9e891de3a51c00c75bcaf192db2f689a34e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-mds-k, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240430, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-05-03T19:05:26.211 DEBUG:teuthology.orchestra.run.smithi045:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f500455e-097b-11ef-bc97-c7b262605968@mds.k.service' 2024-05-03T19:05:26.243 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-03T19:05:26.243 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-05-03T19:05:26.243 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-05-03T19:05:26.243 DEBUG:teuthology.orchestra.run.smithi113:> sudo systemctl stop ceph-f500455e-097b-11ef-bc97-c7b262605968@mds.c 2024-05-03T19:05:26.546 INFO:journalctl@ceph.mds.c.smithi113.stdout:May 03 19:05:26 smithi113 systemd[1]: Stopping Ceph mds.c for f500455e-097b-11ef-bc97-c7b262605968... 2024-05-03T19:05:26.801 INFO:journalctl@ceph.mds.c.smithi113.stdout:May 03 19:05:26 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-mds-c[64907]: 2024-05-03T19:05:26.544+0000 7f1e06a5f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-03T19:05:26.801 INFO:journalctl@ceph.mds.c.smithi113.stdout:May 03 19:05:26 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-mds-c[64907]: 2024-05-03T19:05:26.544+0000 7f1e06a5f700 -1 mds.c *** got signal Terminated *** 2024-05-03T19:05:30.454 INFO:journalctl@ceph.mds.c.smithi113.stdout:May 03 19:05:30 smithi113 podman[70947]: 2024-05-03 19:05:30.152112245 +0000 UTC m=+3.711234455 container died f3eae32eb8de23b8540fe386f924fb8af776c27337754ec1da616ac117a51584 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-mds-c, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.build-date=20240430, io.buildah.version=1.33.5, ceph=True) 2024-05-03T19:05:30.771 INFO:journalctl@ceph.mds.c.smithi113.stdout:May 03 19:05:30 smithi113 podman[70947]: 2024-05-03 19:05:30.5442299 +0000 UTC m=+4.103352115 container cleanup f3eae32eb8de23b8540fe386f924fb8af776c27337754ec1da616ac117a51584 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-mds-c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240430, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-05-03T19:05:30.771 INFO:journalctl@ceph.mds.c.smithi113.stdout:May 03 19:05:30 smithi113 bash[70947]: ceph-f500455e-097b-11ef-bc97-c7b262605968-mds-c 2024-05-03T19:05:30.889 DEBUG:teuthology.orchestra.run.smithi113:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f500455e-097b-11ef-bc97-c7b262605968@mds.c.service' 2024-05-03T19:05:30.925 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-03T19:05:30.925 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-05-03T19:05:30.925 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-05-03T19:05:30.925 DEBUG:teuthology.orchestra.run.smithi113:> sudo systemctl stop ceph-f500455e-097b-11ef-bc97-c7b262605968@mds.f 2024-05-03T19:05:31.051 INFO:journalctl@ceph.mds.f.smithi113.stdout:May 03 19:05:31 smithi113 systemd[1]: Stopping Ceph mds.f for f500455e-097b-11ef-bc97-c7b262605968... 2024-05-03T19:05:31.551 INFO:journalctl@ceph.mds.f.smithi113.stdout:May 03 19:05:31 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-mds-f[64599]: 2024-05-03T19:05:31.247+0000 7f44cab7f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-03T19:05:31.551 INFO:journalctl@ceph.mds.f.smithi113.stdout:May 03 19:05:31 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-mds-f[64599]: 2024-05-03T19:05:31.247+0000 7f44cab7f700 -1 mds.f *** got signal Terminated *** 2024-05-03T19:05:34.551 INFO:journalctl@ceph.mds.f.smithi113.stdout:May 03 19:05:34 smithi113 podman[71079]: 2024-05-03 19:05:34.153364456 +0000 UTC m=+3.009247429 container died 33c4af011e7d40c415be21e9fb808310acc03c4034b42cd17f0378ebcd36518d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-mds-f, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240430, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-05-03T19:05:34.857 INFO:journalctl@ceph.mds.f.smithi113.stdout:May 03 19:05:34 smithi113 podman[71079]: 2024-05-03 19:05:34.631225427 +0000 UTC m=+3.487108408 container cleanup 33c4af011e7d40c415be21e9fb808310acc03c4034b42cd17f0378ebcd36518d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-mds-f, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240430, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-05-03T19:05:34.858 INFO:journalctl@ceph.mds.f.smithi113.stdout:May 03 19:05:34 smithi113 bash[71079]: ceph-f500455e-097b-11ef-bc97-c7b262605968-mds-f 2024-05-03T19:05:34.990 DEBUG:teuthology.orchestra.run.smithi113:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f500455e-097b-11ef-bc97-c7b262605968@mds.f.service' 2024-05-03T19:05:35.022 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-03T19:05:35.022 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-05-03T19:05:35.023 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-05-03T19:05:35.023 DEBUG:teuthology.orchestra.run.smithi113:> sudo systemctl stop ceph-f500455e-097b-11ef-bc97-c7b262605968@mds.i 2024-05-03T19:05:35.301 INFO:journalctl@ceph.mds.i.smithi113.stdout:May 03 19:05:35 smithi113 systemd[1]: Stopping Ceph mds.i for f500455e-097b-11ef-bc97-c7b262605968... 2024-05-03T19:05:35.801 INFO:journalctl@ceph.mds.i.smithi113.stdout:May 03 19:05:35 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-mds-i[65223]: 2024-05-03T19:05:35.341+0000 7f15aa15e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-03T19:05:35.801 INFO:journalctl@ceph.mds.i.smithi113.stdout:May 03 19:05:35 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-mds-i[65223]: 2024-05-03T19:05:35.341+0000 7f15aa15e700 -1 mds.i *** got signal Terminated *** 2024-05-03T19:05:38.394 INFO:journalctl@ceph.mds.i.smithi113.stdout:May 03 19:05:38 smithi113 podman[71199]: 2024-05-03 19:05:38.142585564 +0000 UTC m=+2.906966105 container died e79590a06bad825829089a598f2f37de964a9edb95ace200ee63454647278647 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-mds-i, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240430, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe) 2024-05-03T19:05:38.661 INFO:journalctl@ceph.mds.i.smithi113.stdout:May 03 19:05:38 smithi113 podman[71199]: 2024-05-03 19:05:38.484784263 +0000 UTC m=+3.249164844 container cleanup e79590a06bad825829089a598f2f37de964a9edb95ace200ee63454647278647 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-mds-i, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240430, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-05-03T19:05:38.661 INFO:journalctl@ceph.mds.i.smithi113.stdout:May 03 19:05:38 smithi113 bash[71199]: ceph-f500455e-097b-11ef-bc97-c7b262605968-mds-i 2024-05-03T19:05:38.913 INFO:journalctl@ceph.mds.i.smithi113.stdout:May 03 19:05:38 smithi113 podman[71214]: 2024-05-03 19:05:38.660710717 +0000 UTC m=+0.515816070 container remove e79590a06bad825829089a598f2f37de964a9edb95ace200ee63454647278647 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-mds-i, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240430, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240501.193033-9a047fe) 2024-05-03T19:05:38.917 DEBUG:teuthology.orchestra.run.smithi113:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f500455e-097b-11ef-bc97-c7b262605968@mds.i.service' 2024-05-03T19:05:38.948 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-03T19:05:38.948 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-05-03T19:05:38.949 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-05-03T19:05:38.949 DEBUG:teuthology.orchestra.run.smithi113:> sudo systemctl stop ceph-f500455e-097b-11ef-bc97-c7b262605968@mds.l 2024-05-03T19:05:39.272 INFO:journalctl@ceph.mds.l.smithi113.stdout:May 03 19:05:39 smithi113 systemd[1]: Stopping Ceph mds.l for f500455e-097b-11ef-bc97-c7b262605968... 2024-05-03T19:05:39.551 INFO:journalctl@ceph.mds.l.smithi113.stdout:May 03 19:05:39 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-mds-l[64287]: 2024-05-03T19:05:39.270+0000 7f7fbebf5700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-03T19:05:39.552 INFO:journalctl@ceph.mds.l.smithi113.stdout:May 03 19:05:39 smithi113 ceph-f500455e-097b-11ef-bc97-c7b262605968-mds-l[64287]: 2024-05-03T19:05:39.270+0000 7f7fbebf5700 -1 mds.l *** got signal Terminated *** 2024-05-03T19:05:42.464 INFO:journalctl@ceph.mds.l.smithi113.stdout:May 03 19:05:42 smithi113 podman[71323]: 2024-05-03 19:05:42.143258268 +0000 UTC m=+2.977133191 container died 4f9cf271e240b6c89983e04173d825209567bbf7353d4a5d3c75b463157a1c15 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-mds-l, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240430) 2024-05-03T19:05:42.464 INFO:journalctl@ceph.mds.l.smithi113.stdout:May 03 19:05:42 smithi113 podman[71323]: 2024-05-03 19:05:42.463161332 +0000 UTC m=+3.297036249 container cleanup 4f9cf271e240b6c89983e04173d825209567bbf7353d4a5d3c75b463157a1c15 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-mds-l, org.label-schema.build-date=20240430, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux ) 2024-05-03T19:05:42.801 INFO:journalctl@ceph.mds.l.smithi113.stdout:May 03 19:05:42 smithi113 bash[71323]: ceph-f500455e-097b-11ef-bc97-c7b262605968-mds-l 2024-05-03T19:05:42.802 INFO:journalctl@ceph.mds.l.smithi113.stdout:May 03 19:05:42 smithi113 podman[71337]: 2024-05-03 19:05:42.622457329 +0000 UTC m=+0.478466658 container remove 4f9cf271e240b6c89983e04173d825209567bbf7353d4a5d3c75b463157a1c15 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:68e15ae152ef5c805c733e76da8fc1d3dd4ee669d5ba025f457e05725baa24dd, name=ceph-f500455e-097b-11ef-bc97-c7b262605968-mds-l, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240501.193033-9a047fe, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240430, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True) 2024-05-03T19:05:42.837 DEBUG:teuthology.orchestra.run.smithi113:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f500455e-097b-11ef-bc97-c7b262605968@mds.l.service' 2024-05-03T19:05:42.867 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-03T19:05:42.868 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-05-03T19:05:42.868 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid f500455e-097b-11ef-bc97-c7b262605968 --force --keep-logs 2024-05-03T19:05:43.067 INFO:teuthology.orchestra.run.smithi032.stdout:Deleting cluster with fsid: f500455e-097b-11ef-bc97-c7b262605968 2024-05-03T19:05:45.573 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid f500455e-097b-11ef-bc97-c7b262605968 --force --keep-logs 2024-05-03T19:05:45.772 INFO:teuthology.orchestra.run.smithi045.stdout:Deleting cluster with fsid: f500455e-097b-11ef-bc97-c7b262605968 2024-05-03T19:05:48.247 DEBUG:teuthology.orchestra.run.smithi113:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid f500455e-097b-11ef-bc97-c7b262605968 --force --keep-logs 2024-05-03T19:05:48.437 INFO:teuthology.orchestra.run.smithi113.stdout:Deleting cluster with fsid: f500455e-097b-11ef-bc97-c7b262605968 2024-05-03T19:05:50.795 DEBUG:teuthology.orchestra.run.smithi032:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-03T19:05:50.829 DEBUG:teuthology.orchestra.run.smithi045:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-03T19:05:50.862 DEBUG:teuthology.orchestra.run.smithi113:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-03T19:05:50.888 INFO:tasks.cephadm:Archiving crash dumps... 2024-05-03T19:05:50.890 DEBUG:teuthology.misc:Transferring archived files from smithi032:/var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/crash to /home/teuthworker/archive/rishabh-2024-05-03_13:56:19-fs-wip-rishabh-testing-20240501.193033-testing-default-smithi/7687437/remote/smithi032/crash 2024-05-03T19:05:50.891 DEBUG:teuthology.orchestra.run.smithi032:> sudo tar c -f - -C /var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/crash -- . 2024-05-03T19:05:50.929 INFO:teuthology.orchestra.run.smithi032.stderr:tar: /var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/crash: Cannot open: No such file or directory 2024-05-03T19:05:50.930 INFO:teuthology.orchestra.run.smithi032.stderr:tar: Error is not recoverable: exiting now 2024-05-03T19:05:50.931 DEBUG:teuthology.misc:Transferring archived files from smithi045:/var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/crash to /home/teuthworker/archive/rishabh-2024-05-03_13:56:19-fs-wip-rishabh-testing-20240501.193033-testing-default-smithi/7687437/remote/smithi045/crash 2024-05-03T19:05:50.932 DEBUG:teuthology.orchestra.run.smithi045:> sudo tar c -f - -C /var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/crash -- . 2024-05-03T19:05:50.976 INFO:teuthology.orchestra.run.smithi045.stderr:tar: /var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/crash: Cannot open: No such file or directory 2024-05-03T19:05:50.977 INFO:teuthology.orchestra.run.smithi045.stderr:tar: Error is not recoverable: exiting now 2024-05-03T19:05:50.979 DEBUG:teuthology.misc:Transferring archived files from smithi113:/var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/crash to /home/teuthworker/archive/rishabh-2024-05-03_13:56:19-fs-wip-rishabh-testing-20240501.193033-testing-default-smithi/7687437/remote/smithi113/crash 2024-05-03T19:05:50.980 DEBUG:teuthology.orchestra.run.smithi113:> sudo tar c -f - -C /var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/crash -- . 2024-05-03T19:05:51.019 INFO:teuthology.orchestra.run.smithi113.stderr:tar: /var/lib/ceph/f500455e-097b-11ef-bc97-c7b262605968/crash: Cannot open: No such file or directory 2024-05-03T19:05:51.019 INFO:teuthology.orchestra.run.smithi113.stderr:tar: Error is not recoverable: exiting now 2024-05-03T19:05:51.020 INFO:tasks.cephadm:Checking cluster log for badness... 2024-05-03T19:05:51.021 DEBUG:teuthology.orchestra.run.smithi032:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/f500455e-097b-11ef-bc97-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | head -n 1 2024-05-03T19:05:51.059 INFO:tasks.cephadm:Compressing logs... 2024-05-03T19:05:51.059 DEBUG:teuthology.orchestra.run.smithi032:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-03T19:05:51.102 DEBUG:teuthology.orchestra.run.smithi045:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-03T19:05:51.105 DEBUG:teuthology.orchestra.run.smithi113:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-03T19:05:51.132 INFO:teuthology.orchestra.run.smithi032.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-05-03T19:05:51.132 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/f500455e-097b-11ef-bc97-c7b262605968/ceph.log 2024-05-03T19:05:51.132 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/f500455e-097b-11ef-bc97-c7b262605968/ceph.audit.log 2024-05-03T19:05:51.132 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/f500455e-097b-11ef-bc97-c7b262605968/ceph-mds.g.log 2024-05-03T19:05:51.132 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/f500455e-097b-11ef-bc97-c7b262605968/ceph.log: /var/log/ceph/f500455e-097b-11ef-bc97-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/f500455e-097b-11ef-bc97-c7b262605968/ceph-mds.j.log 2024-05-03T19:05:51.132 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/f500455e-097b-11ef-bc97-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/f500455e-097b-11ef-bc97-c7b262605968/ceph-osd.3.log 2024-05-03T19:05:51.133 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/f500455e-097b-11ef-bc97-c7b262605968/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/f500455e-097b-11ef-bc97-c7b262605968/ceph-osd.0.log 2024-05-03T19:05:51.133 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/f500455e-097b-11ef-bc97-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/f500455e-097b-11ef-bc97-c7b262605968/ceph-mgr.x.log 2024-05-03T19:05:51.133 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/f500455e-097b-11ef-bc97-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/f500455e-097b-11ef-bc97-c7b262605968/ceph-mon.a.log 2024-05-03T19:05:51.133 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/f500455e-097b-11ef-bc97-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/f500455e-097b-11ef-bc97-c7b262605968/ceph-osd.6.log 2024-05-03T19:05:51.133 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/f500455e-097b-11ef-bc97-c7b262605968/ceph-mon.a.log: 88.9% -- replaced with /var/log/ceph/f500455e-097b-11ef-bc97-c7b262605968/ceph.log.gz 2024-05-03T19:05:51.133 INFO:teuthology.orchestra.run.smithi032.stderr: 90.4% -- replaced with /var/log/ceph/f500455e-097b-11ef-bc97-c7b262605968/ceph.audit.log.gz 2024-05-03T19:05:51.133 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/f500455e-097b-11ef-bc97-c7b262605968/ceph.cephadm.log 2024-05-03T19:05:51.134 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/f500455e-097b-11ef-bc97-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/f500455e-097b-11ef-bc97-c7b262605968/ceph-mds.a.log 2024-05-03T19:05:51.134 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/f500455e-097b-11ef-bc97-c7b262605968/ceph.cephadm.log: 82.9% -- replaced with /var/log/ceph/f500455e-097b-11ef-bc97-c7b262605968/ceph.cephadm.log.gz 2024-05-03T19:05:51.138 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/f500455e-097b-11ef-bc97-c7b262605968/ceph-volume.log 2024-05-03T19:05:51.138 INFO:teuthology.orchestra.run.smithi032.stderr: 94.6%/var/log/ceph/f500455e-097b-11ef-bc97-c7b262605968/ceph-mds.a.log: -- replaced with /var/log/ceph/f500455e-097b-11ef-bc97-c7b262605968/ceph-mds.g.log.gz 2024-05-03T19:05:51.138 INFO:teuthology.orchestra.run.smithi032.stderr: 94.6% -- replaced with /var/log/ceph/f500455e-097b-11ef-bc97-c7b262605968/ceph-mds.j.log.gz 2024-05-03T19:05:51.145 INFO:teuthology.orchestra.run.smithi032.stderr: 94.7% -- replaced with /var/log/ceph/f500455e-097b-11ef-bc97-c7b262605968/ceph-mds.a.log.gz 2024-05-03T19:05:51.145 INFO:teuthology.orchestra.run.smithi032.stderr:gzip -5 --verbose -- /var/log/ceph/f500455e-097b-11ef-bc97-c7b262605968/ceph-mds.d.log 2024-05-03T19:05:51.145 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/f500455e-097b-11ef-bc97-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/f500455e-097b-11ef-bc97-c7b262605968/ceph-osd.9.log 2024-05-03T19:05:51.145 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/f500455e-097b-11ef-bc97-c7b262605968/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-03T19:05:51.148 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/f500455e-097b-11ef-bc97-c7b262605968/ceph-osd.9.log: 94.6% -- replaced with /var/log/ceph/f500455e-097b-11ef-bc97-c7b262605968/ceph-mds.d.log.gz 2024-05-03T19:05:51.154 INFO:teuthology.orchestra.run.smithi045.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/f500455e-097b-11ef-bc97-c7b262605968/ceph.log 2024-05-03T19:05:51.154 INFO:teuthology.orchestra.run.smithi045.stderr:: No such file or directory 2024-05-03T19:05:51.154 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/f500455e-097b-11ef-bc97-c7b262605968/ceph.audit.log 2024-05-03T19:05:51.154 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/f500455e-097b-11ef-bc97-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/f500455e-097b-11ef-bc97-c7b262605968/ceph-osd.7.log 2024-05-03T19:05:51.155 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/f500455e-097b-11ef-bc97-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/f500455e-097b-11ef-bc97-c7b262605968/ceph-mds.b.log 2024-05-03T19:05:51.155 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/f500455e-097b-11ef-bc97-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/f500455e-097b-11ef-bc97-c7b262605968/ceph-osd.4.log 2024-05-03T19:05:51.155 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/f500455e-097b-11ef-bc97-c7b262605968/ceph-osd.1.log 2024-05-03T19:05:51.155 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/f500455e-097b-11ef-bc97-c7b262605968/ceph-mds.b.log: /var/log/ceph/f500455e-097b-11ef-bc97-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/f500455e-097b-11ef-bc97-c7b262605968/ceph-mgr.y.log 2024-05-03T19:05:51.155 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/f500455e-097b-11ef-bc97-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/f500455e-097b-11ef-bc97-c7b262605968/ceph-mds.h.log 2024-05-03T19:05:51.155 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/f500455e-097b-11ef-bc97-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/f500455e-097b-11ef-bc97-c7b262605968/ceph-osd.10.log 2024-05-03T19:05:51.155 INFO:teuthology.orchestra.run.smithi045.stderr: 89.0% 90.6% -- replaced with /var/log/ceph/f500455e-097b-11ef-bc97-c7b262605968/ceph.audit.log.gz 2024-05-03T19:05:51.156 INFO:teuthology.orchestra.run.smithi045.stderr: -- replaced with /var/log/ceph/f500455e-097b-11ef-bc97-c7b262605968/ceph.log.gz 2024-05-03T19:05:51.156 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/f500455e-097b-11ef-bc97-c7b262605968/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/f500455e-097b-11ef-bc97-c7b262605968/ceph.cephadm.log 2024-05-03T19:05:51.156 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/f500455e-097b-11ef-bc97-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/f500455e-097b-11ef-bc97-c7b262605968/ceph-mds.e.log 2024-05-03T19:05:51.156 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/f500455e-097b-11ef-bc97-c7b262605968/ceph.cephadm.log: 82.4% -- replaced with /var/log/ceph/f500455e-097b-11ef-bc97-c7b262605968/ceph.cephadm.log.gz 2024-05-03T19:05:51.156 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/f500455e-097b-11ef-bc97-c7b262605968/ceph-volume.log 2024-05-03T19:05:51.156 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/f500455e-097b-11ef-bc97-c7b262605968/ceph-mds.e.log: 93.1% -- replaced with /var/log/ceph/f500455e-097b-11ef-bc97-c7b262605968/ceph-mgr.y.log.gz 2024-05-03T19:05:51.156 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/f500455e-097b-11ef-bc97-c7b262605968/ceph-mon.b.log 2024-05-03T19:05:51.157 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/f500455e-097b-11ef-bc97-c7b262605968/ceph-volume.log: 94.5% -- replaced with /var/log/ceph/f500455e-097b-11ef-bc97-c7b262605968/ceph-mds.e.log.gz 2024-05-03T19:05:51.157 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/f500455e-097b-11ef-bc97-c7b262605968/ceph-mds.k.log 2024-05-03T19:05:51.157 INFO:teuthology.orchestra.run.smithi113.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/f500455e-097b-11ef-bc97-c7b262605968/ceph.log 2024-05-03T19:05:51.157 INFO:teuthology.orchestra.run.smithi113.stderr:: No such file or directory 2024-05-03T19:05:51.157 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/f500455e-097b-11ef-bc97-c7b262605968/ceph-osd.2.log 2024-05-03T19:05:51.157 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/f500455e-097b-11ef-bc97-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/f500455e-097b-11ef-bc97-c7b262605968/ceph-mds.f.log 2024-05-03T19:05:51.158 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/f500455e-097b-11ef-bc97-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/f500455e-097b-11ef-bc97-c7b262605968/ceph.audit.log 2024-05-03T19:05:51.158 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/f500455e-097b-11ef-bc97-c7b262605968/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/f500455e-097b-11ef-bc97-c7b262605968/ceph-osd.5.log 2024-05-03T19:05:51.158 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/f500455e-097b-11ef-bc97-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/f500455e-097b-11ef-bc97-c7b262605968/ceph-mds.c.log 2024-05-03T19:05:51.158 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/f500455e-097b-11ef-bc97-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/f500455e-097b-11ef-bc97-c7b262605968/ceph-mon.c.log 2024-05-03T19:05:51.158 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/f500455e-097b-11ef-bc97-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/f500455e-097b-11ef-bc97-c7b262605968/ceph-mgr.z.log 2024-05-03T19:05:51.158 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/f500455e-097b-11ef-bc97-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/f500455e-097b-11ef-bc97-c7b262605968/ceph-mds.i.log 2024-05-03T19:05:51.158 INFO:teuthology.orchestra.run.smithi113.stderr: 89.0% -- replaced with /var/log/ceph/f500455e-097b-11ef-bc97-c7b262605968/ceph.log.gz 2024-05-03T19:05:51.159 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/f500455e-097b-11ef-bc97-c7b262605968/ceph-mgr.z.log: 90.6% -- replaced with /var/log/ceph/f500455e-097b-11ef-bc97-c7b262605968/ceph.audit.log.gz 2024-05-03T19:05:51.159 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/f500455e-097b-11ef-bc97-c7b262605968/ceph.cephadm.log 2024-05-03T19:05:51.159 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/f500455e-097b-11ef-bc97-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/f500455e-097b-11ef-bc97-c7b262605968/ceph-osd.11.log 2024-05-03T19:05:51.159 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/f500455e-097b-11ef-bc97-c7b262605968/ceph.cephadm.log: 82.3% -- replaced with /var/log/ceph/f500455e-097b-11ef-bc97-c7b262605968/ceph.cephadm.log.gz 2024-05-03T19:05:51.159 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/f500455e-097b-11ef-bc97-c7b262605968/ceph-osd.8.log 2024-05-03T19:05:51.159 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/f500455e-097b-11ef-bc97-c7b262605968/ceph-osd.11.log: 93.2% -- replaced with /var/log/ceph/f500455e-097b-11ef-bc97-c7b262605968/ceph-mgr.z.log.gz 2024-05-03T19:05:51.159 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/f500455e-097b-11ef-bc97-c7b262605968/ceph-volume.log 2024-05-03T19:05:51.159 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/f500455e-097b-11ef-bc97-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/f500455e-097b-11ef-bc97-c7b262605968/ceph-mds.l.log 2024-05-03T19:05:51.161 INFO:teuthology.orchestra.run.smithi032.stderr:/var/log/ceph/cephadm.log: 91.1% -- replaced with /var/log/ceph/f500455e-097b-11ef-bc97-c7b262605968/ceph-volume.log.gz 2024-05-03T19:05:51.162 INFO:teuthology.orchestra.run.smithi032.stderr: 93.6% -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-03T19:05:51.162 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/f500455e-097b-11ef-bc97-c7b262605968/ceph-mon.b.log: 91.1% -- replaced with /var/log/ceph/f500455e-097b-11ef-bc97-c7b262605968/ceph-volume.log.gz 2024-05-03T19:05:51.162 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-03T19:05:51.163 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/f500455e-097b-11ef-bc97-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-03T19:05:51.165 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/f500455e-097b-11ef-bc97-c7b262605968/ceph-mds.k.log: 94.4% -- replaced with /var/log/ceph/f500455e-097b-11ef-bc97-c7b262605968/ceph-mds.k.log.gz 2024-05-03T19:05:51.167 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/cephadm.log: 91.8% -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-03T19:05:51.180 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/f500455e-097b-11ef-bc97-c7b262605968/ceph-mds.l.log: /var/log/ceph/cephadm.log: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-03T19:05:51.180 INFO:teuthology.orchestra.run.smithi113.stderr: 91.0% -- replaced with /var/log/ceph/f500455e-097b-11ef-bc97-c7b262605968/ceph-volume.log.gz 2024-05-03T19:05:51.187 INFO:teuthology.orchestra.run.smithi113.stderr: 94.1% -- replaced with /var/log/ceph/f500455e-097b-11ef-bc97-c7b262605968/ceph-mds.f.log.gz 2024-05-03T19:05:51.207 INFO:teuthology.orchestra.run.smithi113.stderr: 94.0% -- replaced with /var/log/ceph/f500455e-097b-11ef-bc97-c7b262605968/ceph-mds.c.log.gz 2024-05-03T19:05:51.212 INFO:teuthology.orchestra.run.smithi045.stderr: 94.5% -- replaced with /var/log/ceph/f500455e-097b-11ef-bc97-c7b262605968/ceph-mds.h.log.gz 2024-05-03T19:05:51.222 INFO:teuthology.orchestra.run.smithi045.stderr: 94.5% -- replaced with /var/log/ceph/f500455e-097b-11ef-bc97-c7b262605968/ceph-mds.b.log.gz 2024-05-03T19:05:51.244 INFO:teuthology.orchestra.run.smithi113.stderr: 94.6% -- replaced with /var/log/ceph/f500455e-097b-11ef-bc97-c7b262605968/ceph-mds.l.log.gz 2024-05-03T19:05:51.330 INFO:teuthology.orchestra.run.smithi113.stderr: 93.7% -- replaced with /var/log/ceph/f500455e-097b-11ef-bc97-c7b262605968/ceph-mds.i.log.gz 2024-05-03T19:05:51.483 INFO:teuthology.orchestra.run.smithi113.stderr: 92.0% -- replaced with /var/log/ceph/f500455e-097b-11ef-bc97-c7b262605968/ceph-mon.c.log.gz 2024-05-03T19:05:51.486 INFO:teuthology.orchestra.run.smithi032.stderr: 89.8% -- replaced with /var/log/ceph/f500455e-097b-11ef-bc97-c7b262605968/ceph-mgr.x.log.gz 2024-05-03T19:05:51.585 INFO:teuthology.orchestra.run.smithi113.stderr: 92.5% -- replaced with /var/log/ceph/f500455e-097b-11ef-bc97-c7b262605968/ceph-osd.11.log.gz 2024-05-03T19:05:51.588 INFO:teuthology.orchestra.run.smithi045.stderr: 92.0% -- replaced with /var/log/ceph/f500455e-097b-11ef-bc97-c7b262605968/ceph-mon.b.log.gz 2024-05-03T19:05:51.616 INFO:teuthology.orchestra.run.smithi032.stderr: 92.4% -- replaced with /var/log/ceph/f500455e-097b-11ef-bc97-c7b262605968/ceph-osd.0.log.gz 2024-05-03T19:05:51.655 INFO:teuthology.orchestra.run.smithi113.stderr: 92.3% -- replaced with /var/log/ceph/f500455e-097b-11ef-bc97-c7b262605968/ceph-osd.8.log.gz 2024-05-03T19:05:51.666 INFO:teuthology.orchestra.run.smithi045.stderr: 92.5% -- replaced with /var/log/ceph/f500455e-097b-11ef-bc97-c7b262605968/ceph-osd.7.log.gz 2024-05-03T19:05:51.685 INFO:teuthology.orchestra.run.smithi045.stderr: 92.2% -- replaced with /var/log/ceph/f500455e-097b-11ef-bc97-c7b262605968/ceph-osd.4.log.gz 2024-05-03T19:05:51.705 INFO:teuthology.orchestra.run.smithi032.stderr: 92.0% -- replaced with /var/log/ceph/f500455e-097b-11ef-bc97-c7b262605968/ceph-osd.9.log.gz 2024-05-03T19:05:51.738 INFO:teuthology.orchestra.run.smithi113.stderr: 92.4% -- replaced with /var/log/ceph/f500455e-097b-11ef-bc97-c7b262605968/ceph-osd.2.log.gz 2024-05-03T19:05:51.797 INFO:teuthology.orchestra.run.smithi032.stderr: 92.4% -- replaced with /var/log/ceph/f500455e-097b-11ef-bc97-c7b262605968/ceph-osd.3.log.gz 2024-05-03T19:05:51.808 INFO:teuthology.orchestra.run.smithi045.stderr: 92.4% -- replaced with /var/log/ceph/f500455e-097b-11ef-bc97-c7b262605968/ceph-osd.1.log.gz 2024-05-03T19:05:51.895 INFO:teuthology.orchestra.run.smithi032.stderr: 92.2% -- replaced with /var/log/ceph/f500455e-097b-11ef-bc97-c7b262605968/ceph-osd.6.log.gz 2024-05-03T19:05:51.931 INFO:teuthology.orchestra.run.smithi045.stderr: 92.5% -- replaced with /var/log/ceph/f500455e-097b-11ef-bc97-c7b262605968/ceph-osd.10.log.gz 2024-05-03T19:05:51.931 INFO:teuthology.orchestra.run.smithi113.stderr: 92.3% -- replaced with /var/log/ceph/f500455e-097b-11ef-bc97-c7b262605968/ceph-osd.5.log.gz 2024-05-03T19:05:51.934 INFO:teuthology.orchestra.run.smithi113.stderr: 2024-05-03T19:05:51.934 INFO:teuthology.orchestra.run.smithi113.stderr:real 0m0.814s 2024-05-03T19:05:51.934 INFO:teuthology.orchestra.run.smithi113.stderr:user 0m3.035s 2024-05-03T19:05:51.934 INFO:teuthology.orchestra.run.smithi113.stderr:sys 0m0.121s 2024-05-03T19:05:51.934 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-05-03T19:05:51.935 INFO:teuthology.orchestra.run.smithi045.stderr:real 0m0.813s 2024-05-03T19:05:51.935 INFO:teuthology.orchestra.run.smithi045.stderr:user 0m3.084s 2024-05-03T19:05:51.935 INFO:teuthology.orchestra.run.smithi045.stderr:sys 0m0.156s 2024-05-03T19:05:52.058 INFO:teuthology.orchestra.run.smithi032.stderr: 90.3% -- replaced with /var/log/ceph/f500455e-097b-11ef-bc97-c7b262605968/ceph-mon.a.log.gz 2024-05-03T19:05:52.060 INFO:teuthology.orchestra.run.smithi032.stderr: 2024-05-03T19:05:52.060 INFO:teuthology.orchestra.run.smithi032.stderr:real 0m0.944s 2024-05-03T19:05:52.060 INFO:teuthology.orchestra.run.smithi032.stderr:user 0m3.724s 2024-05-03T19:05:52.060 INFO:teuthology.orchestra.run.smithi032.stderr:sys 0m0.125s 2024-05-03T19:05:52.061 INFO:tasks.cephadm:Archiving logs... 2024-05-03T19:05:52.061 DEBUG:teuthology.misc:Transferring archived files from smithi032:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-05-03_13:56:19-fs-wip-rishabh-testing-20240501.193033-testing-default-smithi/7687437/remote/smithi032/log 2024-05-03T19:05:52.062 DEBUG:teuthology.orchestra.run.smithi032:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-03T19:05:52.659 DEBUG:teuthology.misc:Transferring archived files from smithi045:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-05-03_13:56:19-fs-wip-rishabh-testing-20240501.193033-testing-default-smithi/7687437/remote/smithi045/log 2024-05-03T19:05:52.660 DEBUG:teuthology.orchestra.run.smithi045:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-03T19:05:53.139 DEBUG:teuthology.misc:Transferring archived files from smithi113:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-05-03_13:56:19-fs-wip-rishabh-testing-20240501.193033-testing-default-smithi/7687437/remote/smithi113/log 2024-05-03T19:05:53.141 DEBUG:teuthology.orchestra.run.smithi113:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-03T19:05:53.607 INFO:tasks.cephadm:Removing cluster... 2024-05-03T19:05:53.607 DEBUG:teuthology.orchestra.run.smithi032:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid f500455e-097b-11ef-bc97-c7b262605968 --force 2024-05-03T19:05:53.807 INFO:teuthology.orchestra.run.smithi032.stdout:Deleting cluster with fsid: f500455e-097b-11ef-bc97-c7b262605968 2024-05-03T19:05:54.078 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid f500455e-097b-11ef-bc97-c7b262605968 --force 2024-05-03T19:05:54.278 INFO:teuthology.orchestra.run.smithi045.stdout:Deleting cluster with fsid: f500455e-097b-11ef-bc97-c7b262605968 2024-05-03T19:05:54.547 DEBUG:teuthology.orchestra.run.smithi113:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid f500455e-097b-11ef-bc97-c7b262605968 --force 2024-05-03T19:05:54.734 INFO:teuthology.orchestra.run.smithi113.stdout:Deleting cluster with fsid: f500455e-097b-11ef-bc97-c7b262605968 2024-05-03T19:05:54.985 INFO:tasks.cephadm:Removing cephadm ... 2024-05-03T19:05:54.986 DEBUG:teuthology.orchestra.run.smithi032:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-03T19:05:55.003 DEBUG:teuthology.orchestra.run.smithi045:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-03T19:05:55.019 DEBUG:teuthology.orchestra.run.smithi113:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-03T19:05:55.034 INFO:tasks.cephadm:Teardown complete 2024-05-03T19:05:55.035 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-05-03T19:05:55.045 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-05-03T19:05:55.045 DEBUG:teuthology.orchestra.run.smithi032:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-03T19:05:55.048 DEBUG:teuthology.orchestra.run.smithi045:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-03T19:05:55.062 DEBUG:teuthology.orchestra.run.smithi113:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-03T19:05:55.159 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-05-03T19:05:55.172 INFO:teuthology.task.clock:Checking final clock skew... 2024-05-03T19:05:55.172 DEBUG:teuthology.orchestra.run.smithi032:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-03T19:05:55.175 DEBUG:teuthology.orchestra.run.smithi045:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-03T19:05:55.177 DEBUG:teuthology.orchestra.run.smithi113:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-03T19:05:55.188 INFO:teuthology.orchestra.run.smithi032.stderr:bash: line 1: ntpq: command not found 2024-05-03T19:05:55.189 INFO:teuthology.orchestra.run.smithi045.stderr:bash: line 1: ntpq: command not found 2024-05-03T19:05:55.191 INFO:teuthology.orchestra.run.smithi113.stderr:bash: line 1: ntpq: command not found 2024-05-03T19:05:55.193 INFO:teuthology.orchestra.run.smithi032.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-03T19:05:55.193 INFO:teuthology.orchestra.run.smithi032.stdout:=============================================================================== 2024-05-03T19:05:55.193 INFO:teuthology.orchestra.run.smithi032.stdout:^* hv01.front.sepia.ceph.com 3 6 377 34 +1498us[+1497us] +/- 64ms 2024-05-03T19:05:55.193 INFO:teuthology.orchestra.run.smithi032.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 33 -755us[ -755us] +/- 58ms 2024-05-03T19:05:55.193 INFO:teuthology.orchestra.run.smithi032.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 32 -988us[ -988us] +/- 80ms 2024-05-03T19:05:55.193 INFO:teuthology.orchestra.run.smithi032.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-05-03T19:05:55.195 INFO:teuthology.orchestra.run.smithi045.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-03T19:05:55.195 INFO:teuthology.orchestra.run.smithi045.stdout:=============================================================================== 2024-05-03T19:05:55.196 INFO:teuthology.orchestra.run.smithi045.stdout:^* hv01.front.sepia.ceph.com 3 6 377 31 +1502us[+1508us] +/- 64ms 2024-05-03T19:05:55.196 INFO:teuthology.orchestra.run.smithi045.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 33 -751us[ -745us] +/- 58ms 2024-05-03T19:05:55.196 INFO:teuthology.orchestra.run.smithi045.stdout:^+ hv03.front.sepia.ceph.com 3 7 377 99 -995us[ -979us] +/- 79ms 2024-05-03T19:05:55.196 INFO:teuthology.orchestra.run.smithi045.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-05-03T19:05:55.197 INFO:teuthology.orchestra.run.smithi113.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-03T19:05:55.197 INFO:teuthology.orchestra.run.smithi113.stdout:=============================================================================== 2024-05-03T19:05:55.198 INFO:teuthology.orchestra.run.smithi113.stdout:^* hv01.front.sepia.ceph.com 3 6 377 31 +1572us[+1570us] +/- 64ms 2024-05-03T19:05:55.198 INFO:teuthology.orchestra.run.smithi113.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 32 -673us[ -675us] +/- 58ms 2024-05-03T19:05:55.198 INFO:teuthology.orchestra.run.smithi113.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 30 -911us[ -911us] +/- 80ms 2024-05-03T19:05:55.198 INFO:teuthology.orchestra.run.smithi113.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-05-03T19:05:55.198 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-05-03T19:05:55.211 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-05-03T19:05:55.211 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-05-03T19:05:55.226 DEBUG:teuthology.orchestra.run.smithi032:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-03T19:05:55.311 DEBUG:teuthology.orchestra.run.smithi045:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-03T19:05:55.378 DEBUG:teuthology.orchestra.run.smithi113:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-03T19:05:55.441 DEBUG:teuthology.orchestra.run.smithi032:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-03T19:05:55.470 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-03T19:05:55.470 DEBUG:teuthology.orchestra.run.smithi045:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-03T19:05:55.498 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-03T19:05:55.498 DEBUG:teuthology.orchestra.run.smithi113:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-03T19:05:55.524 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-03T19:05:55.524 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-05-03T19:05:55.535 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-05-03T19:05:55.582 INFO:teuthology.task.internal:Duration was 2185.219721 seconds 2024-05-03T19:05:55.583 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-05-03T19:05:55.592 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-05-03T19:05:55.593 DEBUG:teuthology.orchestra.run.smithi032:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-03T19:05:55.595 DEBUG:teuthology.orchestra.run.smithi045:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-03T19:05:55.598 DEBUG:teuthology.orchestra.run.smithi113:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-03T19:05:55.633 INFO:teuthology.orchestra.run.smithi032.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-03T19:05:55.634 INFO:teuthology.orchestra.run.smithi113.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-03T19:05:55.634 INFO:teuthology.orchestra.run.smithi045.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-03T19:05:56.036 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-05-03T19:05:56.036 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi032.front.sepia.ceph.com 2024-05-03T19:05:56.036 DEBUG:teuthology.orchestra.run.smithi032:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-03T19:05:56.059 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi045.front.sepia.ceph.com 2024-05-03T19:05:56.059 DEBUG:teuthology.orchestra.run.smithi045:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-03T19:05:56.097 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi113.front.sepia.ceph.com 2024-05-03T19:05:56.097 DEBUG:teuthology.orchestra.run.smithi113:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-03T19:05:56.120 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-05-03T19:05:56.121 DEBUG:teuthology.orchestra.run.smithi032:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-03T19:05:56.123 DEBUG:teuthology.orchestra.run.smithi045:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-03T19:05:56.140 DEBUG:teuthology.orchestra.run.smithi113:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-03T19:05:56.187 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-05-03T19:05:56.187 DEBUG:teuthology.orchestra.run.smithi032:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-03T19:05:56.192 DEBUG:teuthology.orchestra.run.smithi045:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-03T19:05:56.207 DEBUG:teuthology.orchestra.run.smithi113:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-03T19:05:56.681 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-05-03T19:05:56.692 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-05-03T19:05:56.692 DEBUG:teuthology.orchestra.run.smithi032:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-03T19:05:56.717 DEBUG:teuthology.orchestra.run.smithi045:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-03T19:05:56.741 DEBUG:teuthology.orchestra.run.smithi113:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-03T19:05:56.766 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-05-03T19:05:56.777 DEBUG:teuthology.orchestra.run.smithi032:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-03T19:05:56.779 DEBUG:teuthology.orchestra.run.smithi045:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-03T19:05:56.784 DEBUG:teuthology.orchestra.run.smithi113:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-03T19:05:56.800 INFO:teuthology.orchestra.run.smithi032.stdout:kernel.core_pattern = core 2024-05-03T19:05:56.804 INFO:teuthology.orchestra.run.smithi045.stdout:kernel.core_pattern = core 2024-05-03T19:05:56.829 INFO:teuthology.orchestra.run.smithi113.stdout:kernel.core_pattern = core 2024-05-03T19:05:56.847 DEBUG:teuthology.orchestra.run.smithi032:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-03T19:05:56.867 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-03T19:05:56.868 DEBUG:teuthology.orchestra.run.smithi045:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-03T19:05:56.881 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-03T19:05:56.882 DEBUG:teuthology.orchestra.run.smithi113:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-03T19:05:56.901 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-03T19:05:56.902 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-05-03T19:05:56.913 INFO:teuthology.task.internal:Transferring archived files... 2024-05-03T19:05:56.914 DEBUG:teuthology.misc:Transferring archived files from smithi032:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-05-03_13:56:19-fs-wip-rishabh-testing-20240501.193033-testing-default-smithi/7687437/remote/smithi032 2024-05-03T19:05:56.914 DEBUG:teuthology.orchestra.run.smithi032:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-03T19:05:56.959 DEBUG:teuthology.misc:Transferring archived files from smithi045:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-05-03_13:56:19-fs-wip-rishabh-testing-20240501.193033-testing-default-smithi/7687437/remote/smithi045 2024-05-03T19:05:56.959 DEBUG:teuthology.orchestra.run.smithi045:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-03T19:05:56.995 DEBUG:teuthology.misc:Transferring archived files from smithi113:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-05-03_13:56:19-fs-wip-rishabh-testing-20240501.193033-testing-default-smithi/7687437/remote/smithi113 2024-05-03T19:05:56.995 DEBUG:teuthology.orchestra.run.smithi113:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-03T19:05:57.029 INFO:teuthology.task.internal:Removing archive directory... 2024-05-03T19:05:57.029 DEBUG:teuthology.orchestra.run.smithi032:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-03T19:05:57.032 DEBUG:teuthology.orchestra.run.smithi045:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-03T19:05:57.034 DEBUG:teuthology.orchestra.run.smithi113:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-03T19:05:57.078 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-05-03T19:05:57.090 INFO:teuthology.task.internal:Not uploading archives. 2024-05-03T19:05:57.091 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-05-03T19:05:57.101 INFO:teuthology.task.internal:Tidying up after the test... 2024-05-03T19:05:57.102 DEBUG:teuthology.orchestra.run.smithi032:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-03T19:05:57.104 DEBUG:teuthology.orchestra.run.smithi045:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-03T19:05:57.106 DEBUG:teuthology.orchestra.run.smithi113:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-03T19:05:57.116 INFO:teuthology.orchestra.run.smithi032.stdout: 262213 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 3 19:05 /home/ubuntu/cephtest 2024-05-03T19:05:57.119 INFO:teuthology.orchestra.run.smithi045.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 3 19:05 /home/ubuntu/cephtest 2024-05-03T19:05:57.132 INFO:teuthology.orchestra.run.smithi113.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 3 19:05 /home/ubuntu/cephtest 2024-05-03T19:05:57.133 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-05-03T19:05:57.144 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-05-03T19:05:57.169 INFO:teuthology.nuke:Checking targets against current locks 2024-05-03T19:05:57.193 DEBUG:teuthology.nuke:shortname: smithi032 2024-05-03T19:05:57.193 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-03T19:05:57.213 DEBUG:teuthology.nuke:shortname: smithi045 2024-05-03T19:05:57.214 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-03T19:05:57.219 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi032.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-05-03_13:56:19-fs-wip-rishabh-testing-20240501.193033-testing-default-smithi/7687437', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-03 18:23:22.550681', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGxUx9zXAps/flzGKXz7J/qzDm7uw39KBO6vFgEzFxa043uqPPDmi+vp6QARnLnCM1JXCxrpJj+oM0z2v6/nYiw='} 2024-05-03T19:05:57.234 DEBUG:teuthology.nuke:shortname: smithi113 2024-05-03T19:05:57.234 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-03T19:05:57.241 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi045.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-05-03_13:56:19-fs-wip-rishabh-testing-20240501.193033-testing-default-smithi/7687437', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-03 18:23:22.551932', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCijT7gUF0xj4eI7WPYnoK1LtlFiIbP3NMtDWTAuZ70sh8iXe/89nzXF5017YKXNP037dwirklibLnlOnEPpI58='} 2024-05-03T19:05:57.258 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi113.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-05-03_13:56:19-fs-wip-rishabh-testing-20240501.193033-testing-default-smithi/7687437', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-03 18:23:22.549246', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKsuRiYKdS/fvUtS0hCqaEOFNbV9STbiDwvuJcW4/SCPcs3UIfEMkqQy13DlSIxj+6j5J5sGEe2ASnnCZadPWGE='} 2024-05-03T19:05:57.269 INFO:teuthology.orchestra.console.smithi032:Power off 2024-05-03T19:05:57.270 DEBUG:teuthology.orchestra.console.smithi032:pexpect command: ipmitool -H smithi032.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-05-03T19:05:57.295 DEBUG:teuthology.orchestra.console.smithi032:power off output: Chassis Power Control: Down/Off 2024-05-03T19:05:57.296 DEBUG:teuthology.orchestra.console.smithi032:pexpect command: ipmitool -H smithi032.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-03T19:05:57.313 INFO:teuthology.orchestra.console.smithi045:Power off 2024-05-03T19:05:57.313 DEBUG:teuthology.orchestra.console.smithi045:pexpect command: ipmitool -H smithi045.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-05-03T19:05:57.326 DEBUG:teuthology.orchestra.console.smithi032:check power output: Chassis Power is on 2024-05-03T19:05:57.337 DEBUG:teuthology.orchestra.console.smithi045:power off output: Chassis Power Control: Down/Off 2024-05-03T19:05:57.337 DEBUG:teuthology.orchestra.console.smithi045:pexpect command: ipmitool -H smithi045.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-03T19:05:57.352 INFO:teuthology.orchestra.console.smithi113:Power off 2024-05-03T19:05:57.352 DEBUG:teuthology.orchestra.console.smithi113:pexpect command: ipmitool -H smithi113.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-05-03T19:05:57.362 DEBUG:teuthology.orchestra.console.smithi045:check power output: Chassis Power is on 2024-05-03T19:05:57.374 DEBUG:teuthology.orchestra.console.smithi113:power off output: Chassis Power Control: Down/Off 2024-05-03T19:05:57.374 DEBUG:teuthology.orchestra.console.smithi113:pexpect command: ipmitool -H smithi113.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-03T19:05:57.397 DEBUG:teuthology.orchestra.console.smithi113:check power output: Chassis Power is on 2024-05-03T19:06:01.327 DEBUG:teuthology.orchestra.console.smithi032:pexpect command: ipmitool -H smithi032.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-03T19:06:01.362 DEBUG:teuthology.orchestra.console.smithi045:pexpect command: ipmitool -H smithi045.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-03T19:06:01.398 DEBUG:teuthology.orchestra.console.smithi113:pexpect command: ipmitool -H smithi113.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-03T19:06:01.438 DEBUG:teuthology.orchestra.console.smithi032:check power output: Chassis Power is on 2024-05-03T19:06:01.472 DEBUG:teuthology.orchestra.console.smithi045:check power output: Chassis Power is on 2024-05-03T19:06:01.508 DEBUG:teuthology.orchestra.console.smithi113:check power output: Chassis Power is on 2024-05-03T19:06:05.442 DEBUG:teuthology.orchestra.console.smithi032:pexpect command: ipmitool -H smithi032.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-03T19:06:05.473 DEBUG:teuthology.orchestra.console.smithi045:pexpect command: ipmitool -H smithi045.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-03T19:06:05.510 DEBUG:teuthology.orchestra.console.smithi113:pexpect command: ipmitool -H smithi113.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-03T19:06:05.552 DEBUG:teuthology.orchestra.console.smithi032:check power output: Chassis Power is off 2024-05-03T19:06:05.583 DEBUG:teuthology.orchestra.console.smithi045:check power output: Chassis Power is off 2024-05-03T19:06:05.621 DEBUG:teuthology.orchestra.console.smithi113:check power output: Chassis Power is off 2024-05-03T19:06:05.654 INFO:teuthology.orchestra.console.smithi032:Power off completed 2024-05-03T19:06:05.685 INFO:teuthology.orchestra.console.smithi045:Power off completed 2024-05-03T19:06:05.722 INFO:teuthology.orchestra.console.smithi113:Power off completed 2024-05-03T19:06:05.828 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/fsstress}} duration: 2185.2197206020355 failure_reason: 'Command failed on smithi032 with status 110: "sudo TESTDIR=/home/ubuntu/cephtest bash -c ''ceph fs subvolumegroup pin cephfs qa random 0.10''"' flavor: default owner: scheduled_rishabh@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=fd182ab35b5649f18da4ca8d5ff715b4 status: fail success: false 2024-05-03T19:06:05.829 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-03T19:06:05.910 INFO:teuthology.run:FAIL